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ABSTRACT 


The  use  of  force  ratios  is  an  extremely  valuable  planning  tool  for  the  U.S.  Army. 
Analysis  of  force  ratios  provides  planning  staffs  and  commanders  an  indication  of  the 
required  combat  power  for  successful  operations.  While  the  use  of  force  ratios  is 
prevalent  for  analyzing  maneuver  force  allocation,  little  effort  has  been  made  to  apply 
this  technique  to  air  defense  operations.  The  initial  attempt  at  force  ratio  analysis  within 
the  air  defense  domain,  Correlation  of  Forces  Air  (COFA),  achieves  limited  success. 
However,  the  COFA  process  suffers  from  several  shortcomings. 

This  research  develops  a  methodology  that  is  a  substantial  improvement  over  COFA 
for  air  defense  planning.  Through  the  use  of  a  designed  experiment,  the  study 
demonstrates  the  possibility  of  modeling  the  modem  air  defense  battle  from  simulation 
data.  The  experiment  produced  two  distinct  response  equations  modeling  the  effects  of 
the  various  weapon  systems  on  determining  the  remaining  air  defense  and  task  force 
assets. 

The  development  of  the  means  to  assess  these  two  success  criteria  serves  as  the  basis 
for  the  construction  of  an  alternative  force  allocation  planning  tool.  This  study 
introduces  a  spreadsheet  based  air  defense  force  allocation  planning  tool.  The  tool 
rapidly  produces  point  estimates  of  the  responses  for  multiple  combinations  of  air 


defense  assets  opposing  an  established  air  threat.  Additionally,  the  tool  generates  the  data 
needed  to  quickly  construct  prediction  intervals  for  each  predicted  response.  These 
capabilities  mark  a  significant  improvement  in  force  ratio  analysis  for  air  defense 
operations.  The  spreadsheet  planning  tool  equips  the  Air  Defense  Artillery  commander 
with  the  information  needed  to  make  crucial  force  allocation  decisions. 

This  research  is  limited  by  the  scope  of  military  operations  and  environment 
considered.  Furthermore,  the  use  of  an  unclassified  simulation  database  precludes 
directly  applying  the  variable  relationships  found  here  to  actual  combat  situations. 
However,  the  most  significant  contribution  of  this  research  is  the  development  of  the  new 
methodology  for  analyzing  the  battlefield.  The  framework  developed  in  this  research,  is 
equally  applicable  to  force  ratio  analysis  for  maneuver  units.  Additional  research  that 
accounts  for  the  limitations  of  this  effort  could  result  in  the  development  of  useful  force 
ratio  planning  tools  for  both  Air  Defense  Artillery  and  maneuver  units. 
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INTRODUCTION 


Background 

Since  the  end  of  the  Cold  War,  the  operational  environment  of  the  United  States 
Army  has  changed  drastically.  The  Cold  War  Army  planned  and  trained  to  defend  U.S. 
vital  interests  and  allies  abroad  from  a  clearly  defined  enemy.  Military  planners 
developed  detailed  war  plans  against  the  back  drop  of  established  operational  theaters. 
Units  trained  on  the  same  ground  they  would  fight. 

All  of  that  has  changed.  The  fall  of  the  Iron  Curtain,  the  reunification  of  Germany, 
and  the  breakup  of  the  former  Soviet  Union,  have  dramatically  altered  the  geopolitical 
climate  of  the  world.  Countries  traditionally  considered  to  be  our  adversaries  are  now  our 
allies.  Once  seen  as  a  largely  defensive  force  operating  in  mature  theaters,  the  U.S. 

Army  is  being  increasingly  called  upon  to  respond  rapidly  and  perform  a  variety  of 
missions  around  the  world.  Now,  commanders  and  staffs  must  analyze  missions  and 
allocate  combat  forces  for  deployments  within  extremely  tight  time  constraints  without 
the  luxury  of  previously  developed  detailed  war  plans. 
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Force  Ratio  Analysis 


Critical  to  the  success  of  any  mission  is  ensuring  that  adequate  combat  power  exists 
to  counter  the  enemy  threat.  Analyzing  “force  ratios”  is  a  widely  used  technique  within 
the  U.S.  Army  to  allocate  units  for  missions.  The  technique  involves  comparing  the 
combat  power  of  friendly  and  enemy  forces  by  using  numeric  assessments  of  the  relative 
strengths  of  units.  Relative  unit  strength  is  dependent  on  both  quality  and  quantity  of 
those  units.  For  example,  three  battalions  of  M60  tanks  do  not  possess  the  same  combat 
power  as  three  battalions  of  Ml  A1  tanks  simply  because  the  Ml  A 1  tank  is  a  more 
modem  and  lethal  weapon  system.  In  order  to  make  valid  a  comparison  of  combat 
power,  a  base  unit  is  selected  from  the  expected  enemy.  Planners  then  make  a  subjective 
evaluation  of  the  combat  power  of  all  other  types  of  units  relative  to  the  base  unit.  Once 
planners  establish  these  values  for  all  units  germane  to  the  mission,  they  multiply  each 
unit’s  value  by  the  number  of  units  occurring.  For  the  enemy  side,  the  number  of  units  is 
based  on  intelligence  estimates.  The  new  values  are  totaled  for  each  side  and  multiplied 
by  the  expected  unit  percent  strength.  Finally,  a  force  ratio  comparison  between  the 
enemy  and  friendly  side  is  made  (United  States  Army  Command  and  General  Staff 
College  [USACGSC],  1993).  Table  1  illustrates  this  technique  for  battalion  size  units 
using  a  BTR  battalion  as  the  base  unit.  See  the  glossary  (Appendix  A)  for  descriptions  of 
military  equipment. 
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Table  1 


Notional  Force  Ratio  Analysis 


Type 

Friendly 

Value 

Num. 

Total 

Type 

Enemy 

Value 

Num. 

Total 

M2  bn 

2.00 

1.00 

2.00 

BTRbn 

1.00 

6.00 

6.00 

M1A1  bn 

3.15 

1.00 

3.15 

BMP-2  bn 

1.80 

3.00 

5.40 

Cav  sqdn 

1.50 

1.00 

1.50 

T-80  bn 

2.90 

4.00 

11.60 

Atk  hel  sqdn 

4.00 

1.00 

4.00 

Div  recon  bn 

1.60 

1.00 

1.60 

AT  bn 

1.00 

1.00 

1.00 

Atk  hel  sqdn 

3.00 

1.00 

3.00 

Subtotal 

10.65 

28.60 

x  %  strength 

0.9 

0.8 

TOTAL 

9.56 

22.8 

Force  ratio  9.56  : 

:  22.8  or  1.0:  2.38 

See  the  glossary  (Appendix  A)  for  an  explanation  of  the  unit  descriptions. 


Analyzing  force  ratios  has  proven  useful  for  force  allocation  and  comparing 
alternative  courses  of  action  for  an  operation.  Historical  evidence  has  shown  that  units 
are  successful  in  conducting  operations  provided  they  possess  an  adequate  force  ratio. 
For  example,  a  unit  can  expect  to  have  at  least  a  50%  success  rate  of  defending  from  a 
fortified  position  against  an  attacking  force,  if  the  defending  force  possesses  better  than  a 
1 :3  friend  to  foe  force  ratio  (USACGSC,  1993).  Based  solely  on  this  criteria,  it  follows 
that  a  course  of  action  that  provides  for  a  more  favorable  force  ratio  throughout  the 
operation  would  be  the  preferred  solution.  Although  frequently  accepted  for  analysis  in 
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the  U.S.  Army,  the  force  ratio  analysis  technique  is  not  without  its  detractors.  In  Army 
magazine,  LTC  Robert  R.  Leonhard  (1996)  argues  that  the  numerically  inferior  force 
have  won  most  historical  battles  due  to  increased  maneuverability,  better  command  and 
control,  and  technological  advantages  among  others.  This  argument  aside,  force  ratio 
analysis  is  still  the  accepted  paradigm  in  U.S.  Army  operational  planning.  In  the  example 
in  Table  1,  the  friendly  force  has  a  1 :2.58  force  ratio.  By  the  force  ratio  methodology,  the 
friendly  force  could  reasonably  expect  to  successfully  defend  against  the  enemy  force. 

Correlation  of  Forces  Air 


While  force  ratios  provided  a  useful  tool  for  maneuver  planning,  no  such  tool  existed 
to  assist  in  the  allocation  of  the  supporting  Air  Defense  Artillery  (ADA)  assets.  In  1992, 
the  4th  Battalion  3rd  Air  Defense  Artillery  Regiment  (1993)  of  the  3rd  Infantry  Division 
(3  ID  (M))  introduced  a  procedure,  called  Correlation  of  Forces  Air  (COFA),  that 
integrated  the  concept  of  force  ratios  into  the  ADA  planning  process.  Created  by  COL 
Michael  Vane,  COFA  proposed  numeric  values  of  relative  combat  power  for  threat 
aircraft  and  friendly  ADA  systems.  The  ADA  battalion’s  planners  multiplied  each  of  the 
threat  aircraft  values  by  the  number  of  aircraft  expected  on  each  avenue  of  approach  into 
the  area  of  operation.  The  planner  then  developed  alternative  friendly  ADA  courses  of 
action  by  arraying  ADA  systems  along  each  avenue  approach  with  enough  combat  power 
to  counter  the  expected  threat.  One  of  the  assumptions  of  the  COFA  technique  was  that  a 
1 :3  force  ratio  of  friendly  ADA  to  threat  aircraft  was  sufficient  for  successful  ADA 
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operations.  Table  2  is  an  example  of  this  technique  being  applied  in  a  notional  air 
defense  situation  for  individual  ADA  weapons  and  aircraft.  Utilizing  the  COFA 
technique,  the  ADA  force  specified,  in  the  example,  would  be  sufficient  to  defend  against 
the  expected  threat  aircraft  attack. 


Table  2 

Notional  COFA  Analysis 


Type 

Friendly 

Value 

Num. 

Total 

Type 

Enemy 

Value 

Num. 

Total 

Stinger 

0.5 

10 

5 

MI-8  HIP 

1.8 

20 

36 

BSFV 

0.6 

8 

4.8 

MI-24  HIND 

2.4 

10 

24 

Avenger 

1.9 

6 

11.4 

TOTAL 

21.2 

60 

Force  ratio 

21.2:60  or  1.0: 

2.83 

See  the  glossary  (Appendix  A)  for  a  description  of  ADA  weapons  and  threat  aircraft 
types. 

While  4-3  ADA  used  COFA  with  good  success,  the  technique  does  have  some 
limitations  and  flaws  as  discussed  below.  For  example,  the  numerical  combat  power 
values  are  the  result  of  a  subjective  assessment  of  the  ADA  and  threat  aircraft  systems  by 
one  person,  the  former  battalion  commander.  They  do  not  reflect  any  in  depth  analysis  of 
the  effectiveness  of  the  systems.  Thus,  the  validity  of  the  numerical  values  is  suspect. 
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Secondly,  the  simple  adding  of  combat  power  values  does  not  take  into  account  any 
overlapping  effects  that  may  exist  between  weapon  systems,  friendly  or  enemy.  U.S. 
Army  ADA  doctrine  specifies  the  mixing  of  ADA  weapon  systems  as  an  employment 
principle  for  operational  planning.  The  doctrine  is  based  on  the  belief  that  employing 
ADA  systems  with  different  capabilities  together  offsets  the  limitations  of  the  individual 
systems;  the  net  result  is  a  more  effective  use  of  ADA  assets  than  employing  systems 
independently  (Headquarters,  Department  of  the  Army  [HQ  DA],  1983).  Likewise,  using 
HIPs  and  HINDs  (Appendix  A)  together  may  prove  more  effective  for  the  enemy  than 
employing  them  independently  of  one  another.  The  COFA  technique  ignores  the 
possibility  of  the  overlapping  effect  of  employing  differing  weapon  systems  together. 

Additionally,  the  COFA  technique  does  not  provide  for  separate  force  ratios  for 
differing  operations.  The  heuristic  is  a  1 :3  force  ratio  of  ADA  systems  to  threat  aircraft  is 
adequate  for  the  successful  conduct  of  any  type  of  ADA  operation.  However,  the 
battlefield  disposition  of  ADA  assets  defending  a  maneuver  force  conducting  a  defensive 
operation  is  entirely  different  than  ADA  assets  assigned  to  protect  a  maneuver  unit 
conducting  an  offensive  mission.  In  a  defensive  scenario,  ADA  weapons  are  usually 
emplaced  in  defilade  positions  which  provide  protection  against  enemy  detection  and 
fires.  In  an  offensive  situation,  ADA  systems  are  exposed  a  great  deal  more  to  enemy  fire 
since  maneuvering  elements  are  far  less  likely  to  be  able  to  utilize  defilade  positions.  The 
increased  vulnerability  ADA  assets  encounter  during  offensive  operations  leads  to  an 
increase  in  battle  damage  sustained.  Thus,  it  appears  obvious  that  a  single  force  ratio 
cannot  be  universally  applied  to  all  tactical  situations. 


6 


Perhaps  the  largest  flaw  of  COFA  analysis  is  its  failure  to  adequately  address 
measure  of  success.  The  basic  premise  of  COFA  is  that  if  a  1 :3  ADA  to  air  threat  force 
ratio  is  achieved,  the  ADA  units  will  succeed.  The  U.S.  Army  does  not  have  an  agreed 
upon  definition  of  success  for  air  defense  operations.  For  example,  a  typical  mission  for 
a  Stinger  platoon  is  to  provide  air  defense  to  a  battalion  task  force  conducting  in  a 
defensive  posture.  U.S.  Army  ADA  doctrine  calls  for  the  platoon  to  provide  air  defense 
coverage  “that  allows  the  defended  unit  to  retain  sufficient  combat  power  for  follow-on 
missions”  (HQ  DA,  1 992).  On  the  other  hand,  the  doctrinal  manual  for  a  battalion  task 
force  specifies  that  “the  ratio  of  friendly  vehicle  to  enemy  aircraft  losses  (fixed-wing 
aircraft  and  attack  helicopter)  does  not  exceed  3:1”  (HQ  DA,  1988b).  Another  definition 
of  success  is  provided  in  the  doctrinal  manual  for  the  infantry  battalion  which  specifies 
that  “friendly  losses  to  air  attack  are  less  than  10  percent”  (HQ  DA,  1988a).  It  is  unclear 
if  “success”  to  COFA  is  the  nebulous  ADA  doctrine  specification  or  the  precise 
definitions  that  either  the  battalion  task  force  or  infantry  battalion  doctrines  provide. 
Furthermore,  none  of  the  “success”  criteria  address  the  preservation  of  ADA  combat 
power  for  future  operations.  Unless  a  commander  is  willing  to  accept  Pyrrhic  victories, 
maintaining  both  the  task  force  and  ADA  combat  power  must  be  included  as  success 
criteria. 

Although  the  COFA  technique  has  flaws,  it  does  provide  the  ADA  battalion  staffs  a 
tool  to  quickly  assess  mission  requirements  and  alternative  courses  of  action. 
Furthermore,  it  is  not  known  whether  or  not  the  simplified  process  that  COFA  uses  to 
view  force  ratios  is  inadequate.  As  mentioned  earlier,  the  COFA  technique  was  used 
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successfully  although  on  a  limited  scale.  COFA  is  limited  to  a  “go /  no-go”  decision.  It 
does  not  provide  the  commander  the  means  to  assess  the  risk  to  his  own  force  associated 
with  destroying  the  enemy  given  a  certain  course  of  action.  Commanders  need  a  means 
to  flexibly  wargame  courses  of  action  and  fully  understand  the  impact  of  their  decisions. 
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REPRESENTING  THE  COFA  PHENOMENON 


The  increase  in  the  number  of  unit  deployments,  coupled  with  the  unpredictable 
nature  of  operational  theaters,  has  had  a  tremendous  impact  on  U.S.  Army  units. 
Commanders  and  staffs  must  quickly  analyze  the  mission  at  hand  and  accurately  allocate 
suitable  resources  for  each  contingency.  In  order  to  allocate  resources  for  contingencies 
and  other  operations,  many  staffs  utilize  force  ratio  analysis. 

While  the  use  of  force  ratios  is  a  well  developed  means  for  maneuver  unit  allocation, 
its  corresponding  air  defense  method,  COFA,  is  in  its  infancy.  Although  already  used 
with  some  success,  the  COFA  technique  suffers  from  several  possible  flaws:  the  validity 
of  the  relative  combat  power  values,  the  exclusion  of  the  possible  overlapping  effect  of 
employing  a  mix  of  diverse  weapon  systems  together,  the  failure  to  account  for  the 
differing  dynamics  of  offensive  and  defensive  operations,  and  the  inadequate  address  of 
success  criteria.  While  all  of  these  areas  warrant  further  consideration,  this  research  will 
focuses  on  exploring  overlapping  effects  of  weapon  systems  employed  together  and 
success  criteria. 
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As  with  most  research  problems,  there  are  several  possible  approaches.  The  most 
obvious  approach  is  to  gather  historical  data  and  draw  conclusions  about  the  capabilities 
of  the  systems  from  the  outcomes  of  battles.  In  fact,  the  widely  accepted  1 :3  ratio  for 
units  in  the  defense  was  garnered  from  historical  analysis  (USACGSC).  While  this 
seems  a  logical  choice,  analysis  of  ADA  weapon  systems  presents  particular  difficulties. 
Although  the  United  States  has  engaged  in  several  conflicts  in  recent  years,  there  has 
been  little  to  no  significant  ADA-to-air  threat  interaction.  Furthermore,  meaningful 
literature  and  data  is  classified  and  not  appropriate  for  this  forum  for  national  security 
interests.  In  our  most  recent  large  scale  combat  deployment  during  the  Persian  Gulf  War, 
the  United  States  deployed  both  the  Stinger  and  Avenger  weapon  systems.  However, 
U.S.  Forward  Area  Air  Defense  (FAAD)  forces  did  not  conduct  a  single  engagement 
against  an  enemy  aircraft  during  the  war.  Moreover,  the  Bradley  Stinger  Fighting 
Vehicle  (BSFV)  has  never  been  deployed  for  combat  operations. 


Analysis  of  Combat  Training  Center  Data 


Another  possible  source  of  data  is  the  U.S.  Army’s  Combat  Training  Centers 
(CTCs).  Army  units  conduct  training  at  three  CTCs:  the  National  Training  Center 
(NTC)  at  Ft.  Irwin,  California,  the  Joint  Readiness  Training  Center  (JRTC)  at  Ft.  Polk, 
Louisiana,  and  the  Combat  Maneuver  Training  Center  (CMTC)  at  Hohenfels,  Germany. 
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Each  of  these  training  centers  provides  realistic  force  on  force  training  against  a  highly 
trained  and  well  equipped  opposing  force  (OPFOR).  OPFOR  units,  practicing  typical 
threat  tactics,  engage  and  are  engaged  by  friendly  units  using  the  Multiple  Integrated 
Laser  Engagement  System  (MILES).  MILES  system  sensors  are  attached  to  soldiers’ 
individual  equipment,  vehicles,  and  weapons,  and  provides  both  visual  and  audible  cues 
to  identify  engaged  or  “destroyed”  equipment  and  personnel.  Trained 
Observer/Controllers  (O/Cs)  monitor  all  operations  ensuring  safety  and  provide  training 
feedback  to  units.  The  CTCs  also  incorporate  elaborate  tracking  and  data  collection 
systems  that  facilitate  the  production  of  detailed  After  Action  Review  products  for  the 
training  units. 

Although  the  CTCs  arguably  provide  the  best  training  Army  units  receive  today,  the 
data  from  CTCs  is  inadequate  to  analyze  ADA  to  air  threat  force  ratios.  The  use  of 
MILES  at  the  CTCs  introduces  several  factors  that  invalidate  any  force  ratio  analysis. 
First,  the  MILES  system  only  approximates  the  characteristics  of  the  each  weapon 
system.  Ranges  and  the  accuracy  of  MILES  weapons  differ  from  those  they  represent. 
For  example,  a  Stinger  missile  has  a  range  in  excess  of  4  kilometers  where  a  MILES 
Stinger  system’s  range  is  significantly  less.  Also,  an  actual  Stinger  is  a  “fire  and  forget” 
weapon  system;  once  the  missile  is  fired,  nothing  further  is  required  of  the  Stinger 
crewman.  With  a  MILES  Stinger,  crewman  must  hold  the  weapon  system  on  target  for 
approximately  six  seconds  in  order  for  the  weapon  to  register  a  “kill”  on  an  aircraft. 
Additionally,  MILES  devices  are  subject  to  degraded  performance  due  to  decreased 
visibility  that  are  not  consistent  with  actual  weapon  system  performance.  MILES 
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weapons  also  require  soldiers  to  have  additional  gunnery  skills  to  ensure  that  each  system 
is  correctly  boresighted  to  target.  The  inaccuracies  of  MILES  also  applies  to  OPFOR 
weapons.  Rotary  wing  aircraft  at  the  CTCs  are  represented  by  visually  modified  U.S. 
aircraft  fitted  with  MILES.  In  addition  to  the  size  and  shape  discrepancies  associated 
with  this,  CTC  OPFOR  aircraft  do  not  possess  a  full  complement  of  threat  aircraft 
weapon  systems.  In  effect,  CTC  engagement  data  do  not  show  the  effectiveness  of 
soldiers  using  their  assigned  weapons,  rather  their  MILES  gunnery  skills  against  a  poorly 
simulated  threat  aircraft. 

In  addition  to  the  problems  that  MILES  presents,  CTC  data  has  other  problems 
associated  with  it.  The  focus  of  the  CTCs  is  to  integrate  all  the  components  of  land  and 
air  warfare  into  a  realistic  battlefield.  As  such,  there  are  a  tremendous  amount  of  factors 
that  influence  any  single  battle.  These  can  include  terrain,  weather,  unit  strength,  morale, 
unit  training  proficiency,  and  the  effects  of  multiple  weapon  systems.  Hence,  there  are 
entirely  too  many  factors  that  are  not  controlled  that  preclude  an  analyst  from  isolating 
the  effects  of  individual  weapons  systems  on  one  another  and  the  enemy.  At  best,  an 
analyst  may  be  able  to  identify  trends  in  unit  performance  based  on  weapon  system  use, 
but  certainly  no  concrete  mathematical  relationships. 
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Computer  Simulation 

The  use  of  computer  simulation  offers  a  viable  solution  for  research  of  force  ratios. 
Computer  simulations  estimate  actual  processes  through  the  use  of  mathematical  models. 
These  models  may  be  substantiated  through  engineering  tests  which  yield  the  probability 
of  hit  (PH)  and  probability  of  kill  (PK)  values  used  for  each  weapon  in  the  simulation. 
This  approach  can  also  extend  to  vehicle  and  personnel  movement  characteristics,  the 
effects  of  terrain  and  weather,  and  other  phenomena. 

From  studying  the  models,  an  analyst  can  draw  conclusions  about  the  behavior  of  the 
studied  systems  in  the  real  world.  One  distinct  advantage  of  simulation  is  that  it  provides 
a  completely  controlled  experimental  environment  for  the  analyst  studying  weapon 
systems.  An  analyst  can  fully  specify  every  detail  of  the  scenario,  the  terrain,  and  the 
weapon  systems  employed.  Because  of  this,  he  can  develop  a  complete  experimental 
design  that  will  reveal  the  effects  of  changing  weapon  systems  and  any  interaction  effects 
he  is  interested  in.  Unlike  the  CTC  cases,  the  simulation  experimenter  can  control 
variables  and  interactions  such  as  terrain,  weather,  morale,  and  unit  training  proficiency 
(Law  &  Kelton,  1991).  Furthermore,  many  simulation  models  include  faster  than  real 
time  execution  capability  that  greatly  speeds  data  collection. 

For  all  of  its  advantages,  a  computer  simulation  does  have  limitations.  Since 
simulation  is  based  on  mathematical  models,  it  is  only  an  approximation  of  the  real 
world,  not  an  exact  duplication  (Law  &  Kelton,  1991).  Weapon  systems  are  represented 
by  algorithms  of  the  their  characteristics.  Simulation  models  typically  only  approximate 
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a  portion  of  the  real  world  system  characteristics,  usually  those  most  interesting  to  the 
model  developers.  Some  of  the  most  common  effects  not  typically  represented  in 
simulation  include  the  variability  of  human  entity  behavior  stemming  from  morale  or 
irrationality.  Thus,  conclusions  drawn  on  the  results  of  simulation  are  only  as  accurate  as 
the  simulation  model  represents  them. 

In  order  to  estimate  the  inherent  variability  in  real  world  processes,  many  simulations 
are  stochastic  in  nature.  By  utilizing  random  variates,  simulation  models  attempt  to 
replicate  processes  that  do  not  always  produce  the  same  result  (Law  &  Kelton,  1991). 

For  example,  in  order  to  represent  weapon  system  engagements,  simulations,  such  as 
Janus,  incorporate  previously  established  Probability  of  Hit  (PH)  and  Probability  of  Kill 
(PK)  data  for  given  weapons.  The  simulation  then  utilizes  a  random  variate,  that 
conforms  to  the  PH  and  PK  data,  in  order  to  determine  the  outcome  of  the  engagement 
(Titan,  1993).  Because  of  the  use  of  random  variates,  an  experimenter  must  make  use  of 
multiple  replications,  of  the  same  experimental  conditions,  in  order  to  balance  the  effects 
of  randomness.  Furthermore,  the  use  of  random  variables  results  in  an  estimation  of 
system  behaviors  that  preclude  the  use  of  simulation  as  a  true  optimization  technique 
(Law  &  Kelton,  1991). 

For  the  purposes  of  this  research,  computer  simulation  appears  to  be  the  best 
alternative  for  developing  and  prototyping  a  methodology  for  investigating  for  COFA 
force  ratios.  The  intent  of  this  research  is  to  use  a  computer  simulation,  in  a  controlled 
experiment,  in  order  to  develop  a  methodology  that  will  allow  commanders  to  evaluate 
force  allocation  courses  of  action.  For  a  range  of  force  mix  packages,  the  tool  will 
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provide  the  ability  to  estimate  success  against  a  projected  enemy  based  on  amount  of  the 
defended  task  force  remaining  and  risk  to  ADA  assets. 
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RESEARCH  SCOPE 


The  consideration  of  the  force  ratios  between  ADA  weapons  and  threat  aircraft  is  a 
huge  topic  area.  To  fully  consider  all  factors  of  this  problem  is  well  beyond  the  scope  of 
this  research  effort.  Eliminating  several  factors  and  controlling  others  reduces  the  size  of 
the  research  area. 

The  modem  battlefield  is  cluttered  with  numerous  weapon  systems,  all  with  varying 
capabilities  and  limitations.  Maneuver  commanders  combine  the  combat  power  of  a 
variety  of  armored  vehicles,  artillery  pieces,  and  individual  weapons  in  the  prosecution  of 
each  battle.  The  interaction  of  all  of  these  weapon  systems  with  an  equally  diverse 
enemy  produces  an  enormously  complex  and  dynamic  battlefield.  Although  these 
weapons,  both  friendly  and  enemy,  may  indeed  have  an  effect  on  the  air  battle,  modeling 
and  controlling  all  of  these  factors  would  produce  a  prohibitively  large  experiment. 
Furthermore,  it  is  unclear  if  any  degree  of  accuracy  would  accompany  an  estimate  of  an 
individual  weapon’s  effects  on  various  targets  or  its  interactions  with  other  weapons. 
Although  many  factors  will  influence  the  air  battle,  it  is  likely  that  the  most  significant 
interactions  in  the  air  battle  will  involve  primarily  ADA  weapons  and  aircraft.  Therefore, 
this  limits  the  extent  of  the  analysis  to  a  defended  battalion  task  force,  static  in  location 
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and  composition.  Air  defense  assets  defend  the  task  force  from  enemy  helicopter  attack. 
The  task  force  does  not  encounter  an  opposing  ground  threat. 

The  number  of  ADA  systems  and  aircraft  considered  must  also  be  limited  in  any 
experimentation.  Within  the  current  U.S.  Army,  there  are  six  fielded  air  defense  systems: 
Patriot,  HAWK,  Chaparral,  Stinger,  Avenger,  and  BSFV.  Inclusion  of  all  of  these 
systems  would  result  in  an  extremely  large  experiment.  The  original  COFA  technique 
only  specified  values  for  Chaparral,  Stinger,  Avenger,  and  the  BSFV.  Therefore,  there 
are  no  established  comparison  values  for  the  Patriot  and  HAWK  missile  systems 
Chaparral  system  is  no  longer  in  use  in  the  active  Army  and  is  being  retired  from  service 
in  the  reserve  components.  Thus,  this  research  focuses  on  the  contributions  of  the 
Stinger,  Avenger,  and  BSFV  weapon  systems  to  the  air  battle. 

The  changing  world  political  climate  has  greatly  altered  perceptions  of  what  can  be 
considered  “threat”  or  “friendly”  aircraft.  During  the  Cold  War  era,  U.S.  soldiers 
distinguished  the  weapons  of  the  former  Soviet  Union  as  “threat”  equipment.  Soldiers 
now  face  the  task  of  training  to  identify  equipment  as  friendly  or  enemy  based  on  the 
current  mission.  It  is  completely  infeasible  to  consider  all  possible  threat  aircraft  in  this 
research.  Since  the  original  COFA  technique  established  values  for  the  HIND  and  the 
HIP,  only  these  aircraft  are  included  in  the  designed  scenarios. 

Another  restriction  on  this  analysis  is  the  type  of  operation  investigated.  ADA  units 
support  maneuver  forces  conducting  several  types  of  operations.  Generally,  maneuver 
force  operations  can  be  classified  as  either  offensive  or  defensive  in  nature.  During 
offensive  operations,  ADA  units  typically  move  with  the  supported  maneuver  force. 
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Both  the  defended  task  force  and  the  ADA  assets  are  far  more  vulnerable  to  enemy  fire 
than  in  defensive  operations.  Both  offensive  and  defensive  operations  need  to  be 
considered  separately  in  order  to  fully  test  the  validity  of  COFA  as  a  force  ratio  tool. 

This  would  necessitate  the  construction  of  two  completely  separate  simulation 
experiments  and  a  tremendous  amount  of  time.  Therefore,  only  ADA  units  supporting 
defensive  operations  are  examined. 

The  dynamics  of  the  modem  military  battlefield  are  further  complicated  by  the 
effects  of  terrain,  weather,  and  time  of  day.  Heavily  vegetated  ground,  hilly  terrain,  and 
urban  areas  all  offer  cover  and  concealment  from  enemy  observation  and  fire.  Since 
many  battlefield  systems  are  restricted  to  line-of-sight  engagements,  the  terrain 
encountered  often  restricts  weapons’  engagement  ranges.  Associated  with  each  unique 
terrain  situation,  a  unique  set  of  weapon  system  restrictions  exists.  To  control  the  effects 
of  terrain  in  this  study,  units  conduct  all  operations  in  a  southwest  North  American  desert 
environment.  Opposing  sides  encounter  wide  open  spaces,  however,  weapon  ranges  are 
reduced  in  some  cases  by  a  series  of  mountain  ranges. 

As  does  terrain,  weather  and  time  of  day  also  have  significant  impacts  on  military 
operations.  Poor  weather  has  severe  implications  for  the  trafficability  of  terrain.  Terrain 
passable  in  fair  weather  often  becomes  impossible  for  units  to  operate  in.  Both  weather 
and  time  of  day  have  an  obvious  impact  on  visibility.  Reduced  visibility  tremendously 
impacts  the  effectiveness  of  many  line  of  sight  weapon  systems,  particularly  those  not 
equipped  with  limited  visibility  siting  devices.  The  maneuverability  of  vehicles, 
especially  aircraft,  is  also  greatly  influenced  by  reduced  visibility.  As  a  control  measure, 


18 


all  operations  conducted  in  this  investigation  occur  during  daylight  hours  and  fair 
weather. 

The  numerous  limitations  imposed  on  this  experiment  clearly  limit  the  applicability 
of  the  results  obtained  beyond  these  restrictive  conditions.  A  far  more  comprehensive 
analysis  is  required  in  order  to  adequately  address  the  numerous  factors  influencing  the 
modem  battlefield.  However,  this  analysis  does  address  those  factors  that  likely  have  the 
most  significant  effect  on  the  air  battle:  the  composition  of  the  defending  ADA  force  and 
the  composition  of  the  attacking  enemy  air  threat.  The  greatest  value  of  this  research  will 
be  the  analytical  methodology  developed  for  examining  battlefield  dynamics,  in 
particular  those  influencing  the  air  battle. 


19 


STUDIED  WEAPON  SYSTEMS 


ADA  Weapon  Systems 

The  Stinger  is  a  shoulder  fired,  anti-aircraft  missile.  The  missile  use  an  infrared 
guidance  system  making  it  a  “fire  and  forget”  weapon.  The  Stinger  weapon  includes  an 
Identification  Friend  or  Foe  (IFF)  transponder  and  weighs  approximately  thirty  five 
pounds  on  the  soldier’s  shoulder.  The  Stinger’s  accuracy  and  range,  four  kilometers, 
make  it  a  formidable  air  defense  weapon  against  enemy  rotary  wing  and  low  flying,  fixed 
wing  aircraft  (HQ  DA,  1984). 

A  Stinger  team  consists  of  two  soldiers:  a  team  chief  and  a  crewman.  Stinger  teams 
are  equipped  with  a  High  Mobility  Military  Wheeled  Vehicle  (HMMWV)  that  transports 
up  to  six  Stinger  missiles.  The  team  must  remove  the  missiles  from  the  vehicle  in  order 
to  engage  aircraft.  In  U.S.  Army  divisional  ADA  battalions,  Stinger  teams  are  organized 
into  platoons,  of  ten  teams  each. 

The  Avenger  weapon  system  is  an  advanced  variant  of  the  Stinger.  The  Avenger 
consists  of  a  HMMWV  with  a  weapon  system  turret  mounted  behind  the  cab  on  the 
vehicle  bed.  On  the  turret,  two  missile  pods  are  mounted,  one  on  either  side.  Each  pod 
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is  capable  of  carrying  four  ready  to  fire  Stinger  missiles.  From  the  turret  or  a  remote 
launch  station,  Avenger  crewmen  fire  the  missiles,  in  rapid  succession,  from  either  a 
stationary  position  or  while  the  vehicle  is  in  motion.  The  Avenger  also  incorporates  a 
Forward  Looking  Infrared  (FLIR)  sensor  that  gives  the  system  the  ability  to  detect, 
identify,  and  engage  targets  in  darkness  or  conditions  of  limited  visibility.  The 
combination  Avenger’s  FLIR,  rapid  firing  rate,  and  fire  on  the  move  capability 
tremendously  enhance  the  lethality  of  the  weapon  system  (United  States  Army  Air 
Defense  Artillery  School  [USAADASCH],  1991). 

Avenger  crews  consist  of  two  soldiers  comprising  a  squad:  a  squad  leader  and  a 
crewman.  The  standard  Avenger  platoons  consists  of  six  Avenger  squads.  Avengers 
platoons  have  the  same  composition  in  both  divisional  and  nondivisional  units 
(USAADASCH,  1991). 

The  Bradley  Stinger  Fighting  Vehicle  (BSFV)  is  an  evolving  weapon  system.  The 
current  system  consists  of  a  M2  Bradley  Fighting  Vehicle  that  carrying  a  traditional  two 
man  Stinger  team  in  its  rear  passenger  area.  The  basic  infantry  M2  Bradley  Fighting 
Vehicle  remains  virtually  unchanged  maintaining  its  25mm  main  gun,  a  TOW  missile 
system,  and  a  7.62mm  coaxial  machine  gun.  In  order  to  engage  an  aircraft,  the  vehicle 
stops  and  the  Stinger  team  dismounts  the  vehicle.  The  BSFV  concept  was  developed  to 
provide  the  divisional  Stinger  teams  the  armored  protection  and  firepower  of  the  Bradley 
and  increase  the  mobility  of  Stinger  teams  supporting  maneuver  operations.  BSFVs  are 
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organized  into  platoons  of  four  systems  each.  The  platoon  also  includes  a  standard 
Bradley  Fighting  Vehicle  for  the  platoon  leader  (HQ,  DA,  1995). 

The  newest  version  of  the  BSFV  is  the  Bradley  Linebacker  which  is  still  in  the 
acquisition  phase.  With  its  first  possible  fielding  to  occur  in  1 997,  the  Linebacker 
consists  of  a  Bradley  Fighting  Vehicle  without  its  organic  TOW  missile.  In  place  of  the 
TOW,  the  Linebacker  has  a  modified  Avenger  missile  pod,  with  a  capacity  of  four 
Stingers  (“Bradley  Linebacker,”  1996).  The  result  is  a  highly  mobile  armored  vehicle 
that  can  engage  aircraft  with  an  effectiveness  similar  to  the  Avenger.  In  addition  to  the 
Stinger  and  Avenger,  this  research  includes  the  Linebacker  system  instead  of  the 
currently  fielded  BSFV. 


U.S.  Army  ADA  Tactical  Employment 

The  scope  of  this  research  is  limited  to  air  defense  in  a  defensive  operational  setting. 
Accordingly,  the  following  discussion  of  air  defense  tactics  is  constrained  to  defensive 
operations.  U.S.  Army  air  defense  tactics  are  based  on  four  basic  principles:  mass,  mix, 
mobility,  and  integration.  Mass,  in  air  defense  terms,  implies  having  sufficient  air 
defense  assets  defeat  the  suspected  enemy  air  threat  (HQ  DA,  1983).  The  principle  of 
mass  is  the  impetus  for  force  ratio  analysis. 

To  achieve  mass  when  employing  Stingers,  BSFVs,  and  Avengers,  unit  commanders 
typically  employ  the  systems  close  enough  together  that  their  engagement  ranges  overlap. 
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Ideally,  they  employ  them  within  mutually  supporting  distance.  Assets  are  considered  to 
be  in  mutual  support  of  one  another  if  they  can  provide  coverage  for  each  other’s  “dead 
space”  (HQ  DA,  1992).  A  simple  definition  of  dead  space  is  an  area,  within  the 
engagement  range  of  a  weapon,  in  which  the  weapon  cannot  engage  targets  due  to  terrain 
masking. 

An  additional  measure  to  achieve  mass,  is  weighting  air  defense  coverage  along  the 
most  likely  enemy  air  avenues  of  approach  (AAAs).  In  other  words,  moving  assets  to  the 
area  of  expected  air  threat  while  accepting  risk  in  areas  less  likely  to  encounter  an  air 
attack.  Commanders  can  further  attempt  to  augment  the  mass  principle  by  placing  ADA 
assets  forward  along  the  AAAs  in  order  to  achieve  an  early  engagement  on  the  enemy 
aircraft.  This  reduces  the  possibility  that  the  aircraft  can  pose  a  threat  to  friendly  units  or 
assets  (HQ  DA,  1992). 

The  principle  of  mix  prescribes  mixing  different  air  defense  assets  in  order  to  offset 
the  limitations  of  each  system  (HQ  DA,  1983).  In  practical  terms,  mixing  Stinger  assets 
with  BSFV  units  or  Avengers  would  satisfy  the  principle  of  mix.  Stinger  units  typically 
fight  from  individual  fighting  positions,  or  foxholes,  constructed  to  provide  cover  from 
enemy  fire  (HQ  DA,  1 984).  The  Stinger  team’s  ability  to  construct  their  own  fighting 
positions  allows  it  to  be  effectively  employed  in  numerous  tactical  situations  (HQ  DA, 
1992). 

Because  of  their  vulnerability  to  damage  from  enemy  fire,  Avengers  are  best  suited 
for  the  defense  of  assets  not  involved  in  direct  ground  combat.  If  used  in  forward  areas, 
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Avengers  should  be  placed  along  the  flanks  of  units  or  in  overwatch  positions 
(USAADASCH,  1991).  On  the  other  hand,  the  armament  of  the  BSFV  provides  a 
measure  of  protection  to  its  crew  and  equipment.  The  BSFV  was  designed  with  the 
support  of  the  forward  maneuver  force  in  mind.  However,  whenever  possible,  BSFV 
platoon  leaders  will  attempt  to  coordinate  for  combat  engineer  assets  to  dig  defilade 
positions  for  their  BSFVs  when  supporting  defensive  operations  (HQ  DA,  1995). 

Mobility  calls  for  ADA  assets  to  be  sufficiently  mobile  to  move  about  the  battlefield 
in  response  to  changing  tactical  conditions.  Additionally,  all  ADA  efforts  must  be  fully 
integrated  into  the  maneuver  scheme  of  operations  in  order  to  insure  success  (HQ  DA, 
1983).  Due  to  the  scope  of  this  research,  the  principles  of  mobility  and  integration  are  not 
fully  addressed  within  this  experiment. 

Threat  Aircraft 


The  Mi-8  HIP  is  a  medium  transport/assault  helicopter.  Available  in  numerous 
configurations,  the  Mi8T  HIP  E  is  the  standard  gunship  version.  It  has  a  swivel  mounted 
12.7  mm  nose  machine  gun,  outrigger  mounted,  six  32  shot  57  mm  rocket  pods,  four  250 
kilogram  bombs,  and  four  AT2C  SWATTER  anti-tank  guided  missiles  (ATGM).  The 
helicopter  is  heavily  armored  and  is  capable  of  transporting  up  to  24  troops  and  1,000 
kilograms  of  ordinance  simultaneously.  The  helicopter  is  also  capable  of  transporting  up 
to  24  troops  for  air  assault  missions.  Produced  by  the  former  Soviet  Union,  the  aircraft 
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has  been  widely  exported  around  the  world  (Headquarters,  Department  of  the  Army  [HQ 
DA], 1991). 

The  Mi-24  HIND  is  one  of  the  most  heavily  armed  attack  helicopters  in  the  world. 
The  HIND  is  available  in  many  variants,  each  with  differing  weapon  systems.  Dubbed 
the  “flying  tank”,  the  HIND  D  features  a  four  barrel  12.7mm  Gatling  type  machine  gun, 
turret  mounted  under  the  nose  of  the  aircraft.  It  also  has  four  32  shot  57mm  rocket  pods, 
and  four  AT2C  Swatter  ATGMs.  In  lieu  of  the  57mm  rocket  pods,  the  HIND  D  is 
capable  of  carrying  250  or  500  kilogram  bombs.  The  helicopter  is  primarily  utilized  in  a 
close  air  support  role  especially  against  enemy  tanks.  The  helicopter  can  also  transport  8 

i 

troops  for  use  in  air  assault  operations  (HQ  DA,  1991).  Produced  by  the  former  Soviet 
Union,  the  helicopter  can  be  found  in  former  Warsaw  Pact  nations  and  in  several 
countries  in  the  Middle  East,  Far  East,  Africa,  and  South  America  (Foreign  Material 
Intelligence  Battalion,  1990). 

Typical  Threat  Helicopter  Tactics 

Defining  threat  helicopter  tactics,  in  the  post  Cold  war  era,  is  an  extremely 
situationally  dependent  venture.  Since  the  scope  of  this  study  is  limited  to  the 
consideration  of  HIPs  and  HINDs,  included  below  is  a  description  of  the  helicopter 
tactics  of  the  former  Soviet  Union.  In  fact,  the  U.S.  Army  Training  and  Doctrine 
Command  (TRADOC)  utilizes  the  Soviet  style  tactics  as  the  basic  building  block  for 
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developing  Opposing  Force  (OPFOR)  tactics  for  the  Combat  Training  Centers. 

TRADOC  believes  that  the  countries  of  the  former  Soviet  Union  and  its  former  allies  will 
continue  to  pattern  their  doctrine  and  tactics  on  those  of  the  former  Soviet  Union  (United 
States  Army  Training  and  Doctrine  Command  [TRADOC],  1994). 

The  threat  typically  uses  attack  helicopters  as  an  extension  of  the  artillery.  These 
“flying  artillery”  platforms  usually  attack  in  a  flight  of  four  aircraft  but  can  also  be 
employed  in  pairs.  Attacks  can  be  made  from  one  or  several  directions  simultaneously. 
Common  attack  profiles  begin  with  a  high  speed,  low  level  flight  ingress  while  firing, 
running  fire.  Once  near  the  target,  the  helicopters  ascend  to  acquire  and  engage  targets. 
Helicopters  engage  using  either  the  running  fire  or  hovering  fire  techniques.  The 
helicopters  then  dive  for  a  low  level  departure  [TRADOC,  1994].  Attack  helicopters 
ingress  and  egress  along  air  avenues  of  approach  (AAA).  Typical  AAAs  include  river 
beds  and  valleys  that  provide  terrain  masking  from  enemy  observation.  Helicopters  will 
also  fly  behind  ridge  lines  to  avoid  enemy  fire  (HQ  DA,  1995). 

Although  there  are  numerous  missions  for  threat  attack  helicopters,  the  most 
common  use  is  in  the  direct  air  support  role.  The  mission  is  normally  flown  by  four 
helicopters  attacking  with  rockets  and  guns  or  ATGMs  in  support  of  ground  combat 
operations.  These  attacks  emphasize  limiting  exposure  time  to  about  20  seconds 
primarily  using  a  running  gun  technique  [TRADOC,  1994].  Because  of  its  weight, 
HINDs  cannot  effectively  hover  while  combat  loaded  and  are  therefore  limited  to  the 
running  gun  technique  (HQ  DA,  90). 
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THE  JANUS  SIMULATION  SYSTEM 


Background 

Janus  6.0,  a  constructive  military  simulation,  is  the  simulation  model  for  this 
research.  Janus  is  suitable  for  this  type  of  analysis  since  it  accurately  models  both  rotary 
wing  aircraft  and  air  defense  weapon  systems.  Janus  is  also  widely  used  and  accepted 
throughout  the  military  making  it  readily  accessible  those  wishing  to  confirm,  or  expand 
on,  the  results  of  this  study.  Additionally,  there  exists  validated  data  for  the  modeling  of 
U.S.  military  and  threat  weapon  systems.  Because  the  validated  data  is  classified,  this 
experiment  uses  an  unclassified  training  data  base  for  analysis.  However,  the 
methodology  will  remain  sound  as  well  as  transferable  to  an  experiment  using  classified 
data. 

Janus  was  named  for  the  two  faced  Roman  God  of  portals  to  reflect  the  ability  to  see 
the  battlefield  from  both  sides.  Janus  exists  in  several  different  versions.  The  Lawrence 
Livermore  National  Laboratory  developed  the  initial  version  of  Janus,  Janus  (L),  as  a 
means  to  model  nuclear  effects.  Subsequently,  another  version,  Janus  (T),  was  developed 
for  the  U.S.  Army’s  Training  and  Doctrine  Command  (TRADOC),  Training  and  Analysis 
Center  (TRAC)  to  assist  in  the  combat  development  field.  From  Janus  (T), 
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a  third  version  was  fielded  for  use  in  the  training  community.  The  current  version  used  in 
this  research,  Janus  (Army),  is  commonly  known  as  simply  Janus  (U.S.  Army 
Simulation,  Training  and  Instrumentation  Command  [STRICOM],  undated). 


Janus  is  a  stochastic,  combat  simulation.  The  current  version,  Janus  6.0,  can 
accommodate  up  to  six  opposing  sides.  The  simulation  allows  training  personnel  to 
interact  with  the  systems  modeled  in  the  simulation  during  run  time.  Since  it  is  primarily 
a  training  simulation,  each  opposing  force  only  knows  the  disposition  of  other  forces 
when  its  forces  can  detect  the  opposing  force  in  the  simulation.  Detections  between 
modeled  systems,  weapons  effects  and  engagements  are  determined  by  stochastic 
processes.  Janus  models  maneuver,  artillery,  air  defense,  and  engineer  units,  as  well  as 
minefields,  obstacles,  chemical  effects,  and  aircraft  (STRICOM). 

Janus  uses  digitized  data  from  the  Defense  Mapping  agency  for  its  terrain 
representation.  It  models  the  effects  of  terrain  and  vegetation  on  line  of  sight  and 
mobility.  Janus  also  accounts  for  the  impact  of  weather,  visibility  and  light  data 
(STRICOM). 
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Janus  Output  Measures 


Janus  supports  the  detailed  analysis  of  simulations  runs  through  two  tools:  the  Janus 
Analyst  Workstation  (JAWS)  and  the  Post  Processor.  Janus  records  all  simulation  data  in 
recording  files  for  each  simulation  run.  JAWS  accesses  the  recording  files  and  allows 
playback  of  the  entire  simulation  or  selected  events.  The  user  can  specify  playback 
speeds  equal  to  or  exceeding  real  time.  JAWS  is  particularly  useful  for  conducting  After 
Action  Reviews  (AARs)  of  unit  training  (STRICOM). 

Janus  also  incorporates  a  Post  Processor  that  retrieves  information  from  the 
simulation  recording  files  of  single  or  multiple  simulation  runs.  The  Post  Processor 
presents  the  requested  information  in  report  form.  These  reports  include  information 
regarding  the  effectiveness  and  time  of  engagement  of  indirect  and  direct  fire  weapons. 
Coroner’s  reports  provide  a  detailed  account  of  each  battlefield  kill  in  terms  of  time 
location  and  participants.  Other  reports  show  the  impact  of  minefields,  temperature,  and 
chemical  weapons  on  units.  Game  Analysis  reports  summarize  engagement  and  detection 
ranges  as  well  as  indicate  the  contributions  of  selected  weapon  systems  (STRICOM). 

The  Killer/Victim  Scorecard  is  of  particular  value  to  this  research.  Presented  in 
tabular  form,  the  report  displays  all  possible  combinations  of  “killers”  and  “victims”  in 
the  simulation.  The  report  identifies  the  number  and  type  of  weapon  systems  destroyed, 
“victims”,  by  each  weapon  system  type,  “killers”  (STRICOM).  For  example,  the 
Killer/Victim  Scorecard  will  identify  the  number  of  HIND  helicopters  destroyed  by 
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Stinger  weapon  systems.  Likewise,  it  will  show  the  number  of  Stingers  destroyed  by 


HINDs. 


Aircraft  Modeling 

Like  all  weapon  systems  in  Janus,  Janus  represents  aircraft  through  a  series  of 
mathematical  algorithms.  Weapon  characteristics,  such  as  weapon  range,  weapon 
effectiveness  and  survivability,  are  captured  in  database  format.  Scenario  developers  can 
manipulate  the  database  in  order  to  see  the  effect  of  changes  in  weapon  characteristics  on 
studied  scenarios. 

The  Janus  user  can  specify  several  movement  parameters  for  aircraft.  Janus  aircraft 
movement  routes  can  be  established  prior  to  runtime,  in  the  scenario  planning  phase,  or 
during  scenario  execution.  The  user  may  also  dynamically  alter  routes  during  runtime. 
Besides  direction,  the  user  controls  both  aircraft  altitude  and  speed  through  the  use  of  two 
flight  modes:  “high  and  fast”  or  “low  and  slow”.  The  settings  for  the  altitude  and  speed, 
for  each  flight  mode,  are  contained  in  the  database.  The  user  can  manipulate  these 
settings  prior  to  runtime.  During  execution,  the  user  can  dynamically  change  the 
selection  of  flight  mode.  However,  during  runtime,  he  may  not  change  the  preset 
altitudes  or  speeds  that  correspond  with  each  flight  mode.  Aircraft  routes  may  consist 
entirely  of  “high  and  fast”  or  “low  and  slow”  operations.  Aircraft  routes  may  also  consist 
of  multiple  segments,  each  with  a  different  flight  mode,  “high  and  fast”  or  “low  and 
slow”  (STRICOM). 
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Many  combat  aircraft  also  make  use  of  “pop  up”  tactics.  “Pop  up”  tactics  consist  of 
an  aircraft  hovering  behind  a  terrain  feature,  out  of  the  line  of  enemy  fire.  Aircraft  will 
ascend  to  a  “pop  up”  altitude,  above  the  cover  of  the  terrain,  in  order  to  identify  and 
engage  the  enemy.  Following  the  engagement,  the  aircraft  will  descend  behind  the 
terrain.  Janus  allows  for  the  establishment  of  positions  for  use  as  “pop  up”  locations.  As 
with  flight  modes,  users  can  switch  the  aircraft  from  a  hovering  to  a  “pop  up”  altitude 
during  runtime,  however,  the  altitude  values  are  set  in  the  database  and  not  alterable 
during  scenario  execution  (STRICOM). 

Janus  models  both  helicopters  and  fixed  wing  aircraft  through  the  use  of  the  same 
parameters.  Since  in  most  cases  fixed  wing  aircraft  cannot  hover,  the  database  fields  for 
“pop  up”  data  are  left  blank.  Although  much  has  been  said  about  poor  fixed  wing 
modeling  in  Janus,  research  has  shown  that  Janus  adequately  captures  helicopter  effects 
(Daniels,  1994). 


ADA  Weapon  Modeling 

Due  to  the  complexity  of  many  military  weapons,  Janus  models  many  weapon 
systems  as  compilations  of  several  subsystems.  For  example,  a  U.S.  Ml  A2  Abrams  tank 
will  consist  of  the  carrier  and  two  gun  subsystems:  the  120mm  smooth  bore  gun  and  the 
coaxial  machine  gun.  Since  the  gun  systems  may  be  types  used  by  other  weapon 
systems,  they  each  have  stand  alone  database  entries. 
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Janus  models  the  ADA  weapon  systems  studied  here  using  the  same  compilation 
technique.  The  Stinger  missile  is  the  common  thread  between  all  three  weapon  systems 
and  has  a  separate  entry  in  the  weapon  database.  Although  the  actual  missile  is  the  same 
between  all  three  weapon  systems,  some  of  its  characteristics  change  when  used  in  the 
Avenger  and  Linebacker  systems.  In  the  Janus  database,  values  are  specified  for  reload 
times  and  number  rounds  fired  prior  to  reload.  The  values  are  for  the  Stinger  weapon 
when  utilized  in  a  shoulder  fired  configuration,  valid  for  a  Stinger  team.  In  the  Avenger 
and  Linebacker  configurations,  eight  rounds  and  four  rounds,  respectively,  can  be  fired 
before  reload  occurs  compared  to  one  round  before  reload  for  the  shoulder  fired  Stinger. 
The  difference  in  equipment  and  number  of  rounds  to  reload  also  changes  the  mean 
reload  time  values  for  the  Avenger  and  Linebacker.  Adjustments  were  made  to  the 
database  to  more  accurately  reflect  these  systems’  capabilities  while  keeping  the  actual 
data  and  system  behaviors  unclassified  yet  realistic  for  the  purpose  of  this  research. 
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EXPERIMENTAL  AND  SCENARIO  DESIGN 


Identifying  the  relationships  between  the  factors  influencing  the  air  battle  is  essential 
to  the  development  of  an  alternative  force  ratio  planning  tool.  This  research  focuses  on 
exploring  the  impact  that  the  amount  of  friendly  ADA  systems  and  the  amount  of 
attacking  enemy  aircraft  have  on  the  remaining  combat  strength  of  the  defended  task 
force  and  ADA  assets.  The  research  incorporates  a  standard  factorial  experiment  that 
allows  analysis  of  these  variables  within  a  tactical  scenario. 


Scenario  Design 


This  investigation  is  limited  to  the  analysis  of  ADA  weapon  systems  defending  a  unit 
in  a  defensive  posture.  In  the  scenario,  a  battalion  task  force  consisting  of  three 
mechanized  infantry  companies  and  one  tank  company  is  arrayed  in  a  hasty  defense  on  a 
terrain  database  of  the  National  Training  Center  at  Fort  Irwin,  California..  The  task  force 
is  arrayed  along  a  likely  enemy  avenue  of  approach  capable  of  supporting  a  regimental 
size  threat  advance.  In  order  to  validate  the  task  force’s  defensive  plan,  two  U.S.  Army 
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Infantry  officers  were  consulted:  MAJ  John  McCarthy  and  CPT  Brian  Bedell.  Each 
officer  agreed  that  the  task  force  defense  design  was  both  feasible  and  tactically  sound. 

ADA  assets  are  emplaced  in  support  of  the  task  force  defense  in  accordance  with 
current  U.S  Army  ADA  doctrine.  The  actual  enemy  attacking  the  task  force  consists  of 
the  air  threat  assets  previously  mentioned.  The  aircraft  follow  predetermined  flight 
patterns  and  routes.  Each  flight  pattern  and  route  adheres  to  the  Soviet  style  attack 
helicopter  doctrine  described  previously.  The  scenario  concludes  when  all  aircraft  have 
completed  their  attack  routes  or  are  destroyed. 

The  task  force  does  not  encounter  an  attacking  enemy  ground  force.  The  inclusion  of 
an  enemy  ground  force  would  introduce  additional  challenges  for  controlling  the 
experiment.  The  added  complexity  of  the  experiment  could  cloud  estimation  of  the 
critical  ADA  and  air  threat  factors.  It  is  important  to  understand,  that  it  is  inadvisable  to 
make  inferences  regarding  the  performance  of  ADA  or  air  threat  assets  to  situations  that 
include  enemy  ground  forces.  Any  such  inferences  would  be  based  on  incomplete  data 
and  subject  to  considerable  error. 
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Experimental  Design 


A  traditional  factorial  experimental  design  provides  the  framework  for  analyzing  the 
dynamics  of  the  air  battle.  The  experiment  contains  five  factors  or  independent  variables: 
the  number  of  Stingers  (S),  the  number  of  Bradley  Linebackers  (L),  the  number  of 
Avengers  (A),  the  number  of  HIPs  (Hp),  and  the  number  of  HINDs  (Hd)  (appendix  B). 
The  effects  of  changing  the  levels  of  these  variables  is  measured  against  two  responses  or 
dependent  variables:  the  percent  of  friendly  ADA  systems  remaining  and  the  percent  of 
the  friendly  task  force  remaining. 

The  number  of  ADA  systems  at  the  start  of  each  simulation  run  is  variable  based  on 
the  data  point  examined.  In  simulation  runs  where  no  ADA  systems  are  destroyed, 
scenarios  with  higher  initial  ADA  numbers  would  automatically  result  in  a  higher 
response,  if  the  response  is  defined  as  the  number  of  ADA  weapons  remaining.  Scaling 
the  response  in  terms  of  percentages  eliminates  this  bias.  The  remaining  number  of 
defended  task  force  vehicles  is  not  subject  to  this  bias.  However  for  consistency, 
percentages  are  used  for  this  response  as  well. 

With  a  traditional  factorial  experiment,  each  independent  variable  is  assigned  a 
“high”  and  a  “low”  factor  level.  The  factor  levels  correspond  to  the  amount  or  value  of 
each  independent  variable  used  in  the  experiment.  The  amounts  of  the  independent 
variables  used  in  the  experiment  are  restricted  to  these  factor  levels.  In  this  research,  the 
factor  levels  correspond  to  numbers  of  ADA  weapon  system  and  number  of  enemy 
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aircraft  at  the  start  of  each  simulation  run.  During  experiment,  the  analyst  examines  the 
results  of  all  possible  combinations  of  the  “high”  and  “low”  factor  levels  on  the  selected 
response  variables.  By  analyzing  all  possible  combinations,  the  analyst  can  isolate  the 
effect  that  each  independent  variable  has  on  each  response  variable. 

The  selected  experimental  design  consists  of  1 6  factorial  points  and  one  center  run. 
The  factorial  portion  of  the  design  is  a  resolution  V,  half  fraction  of  the  full  factorial 

design  for  five  factors,  25v' .  In  a  resolution  V  design,  no  main  effect  is  aliased  with  any 
other  main  effect  or  two  factor  interactions,  nor  are  two  factor  interactions  aliased  with 
each  other  (appendix  B).  Thus,  the  design  provides  a  relatively  good  estimate  of  the  main 
effects  and  two  factor  interactions.  Due  to  the  sparsity  of  effects  principle  (Meyers  & 
Montgomery,  1 995)  and  the  nature  of  the  factors  in  this  problem,  it  is  likely  that  the 
responses  will  be  dominated  by  the  main  effects  and  two  factor  interactions. 

For  the  2  *_1  design,  each  factor  is  varied  between  two  levels,  a  high  and  a  low  level. 
The  high  and  low  levels  are  coded  in  the  design  as  +1  for  high  and  -1  for  low.  The  initial 
levels  for  each  factor  in  the  design  are  contained  in  Table  3.  The  selection  of  the  initial 
factor  levels  is  an  arbitrary  decision  based  on  unit  composition.  For  the  ADA  systems, 
the  high  levels  corresponds  to  the  number  of  ADA  systems  found  in  two  platoons  of  each 
system  type.  For  the  aircraft,  the  high  level  corresponds  to  the  typical  number  of  aircraft 
found  in  two  squadrons  of  each  aircraft  type. 
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Table  3 


Initial  Factor  Levels 

Factor  High  Low 

Stingers  (S)  20  0 

Linebackers  (L)  8  0 

Avengers  (A)  12  0 

HIPs  (Hp)  40  0 

HINDs  (Hd)  40  0 

The  25v'  design  factorial  portion  of  the  design  assumes  that  there  are  no  quadratic 
effects  in  the  model.  In  order  to  test  that  assumption,  the  design  includes  center  point. 
The  center  point  allows  testing  for  curvature  in  the  model.  The  Data  Analysis  chapter 
contains  the  results  of  this  test. 

As  stated  previously,  Janus  is  a  stochastic  simulation.  In  order  to  account  for  the 
inherent  variability  of  the  model,  the  design  replicates  each  design  point  several  times  to 
establish  an  estimate  of  the  responses  at  that  design  point.  Randomly  selected  seeds  for 
each  replication  ensure  independence  between  replications. 

To  determine  the  number  of  replications  needed,  the  “specified  precision”  technique 
suggested  by  Law  and  Kelton  (1991)  is  used.  With  the  “specified  precision”  technique, 
the  analyst  specifies  a  maximum  acceptable  absolute  error  on  the  estimate  of  the  mean,  /?. 
For  an  established  /?,  the  analyst  projects  the  number  of  replications  required  based  on  the 
results  of  a  few  initial  replications.  Assuming  that  the  actual  population  variance  does 
not  differ  greatly  from  the  estimate  of  population  variance  based  on  the  sample,  the 
required  number  of  replications  («* )  is  given  by: 
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(1) 


na  =  minjz  >  n\tt_x  x_a/js2  {n)  I  i  <  /?J 
where: 

n  =  number  of  replications  in  the  sample 

i  =  projected  number  of  replications 

S2(n )  =  estimate  of  population  variance  based  on  the  sample 

P  =  \X  -  ju\  =  absolute  error  of  the  estimate 
t  =  critical  value  from  the  Student’s  t  distribution. 


This  experiment  is  exploratory  in  nature  with  the  purpose  of  developing  a 
methodology  for  more  encompassing  research.  Furthermore,  it  is  not  within  the  intent  of 
this  research  to  identify  valid  relationships  between  the  variables  for  direct  application 
outside  of  this  research;  the  election  to  use  an  unclassified  training  database  precludes 
directly  applying  the  analytical  results  of  this  research  in  a  tactical  setting.  Understanding 
these  intentions,  80%  is  a  reasonable  level  of  precision  for  confidence  intervals  on  the 
estimate  of  the  mean  responses. 

The  selection  of  (3  for  an  experiment  is  an  arbitrary  decision  of  the  analyst.  In  this 
experiment,  the  defended  task  force  consists  of  slightly  over  50  vehicles.  The  maximum 
number  of  ADA  systems  employed  at  any  design  point  is  40.  Based  on  the  magnitude  of 
these  numbers,  an  estimate  of  the  remaining  task  force  vehicles  or  ADA  systems  that  is 
within  1 .5  vehicles  of  the  actual  mean  seems  accurate  enough  for  this  research. 

Therefore,  for  80%  of  the  confidence  intervals  on  the  means  that  are  constructed,  the 
expected  absolute  error  is  1.5  vehicles. 
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When  utilizing  the  “specified  precision”  technique,  increased  precision  results  in  a 
corresponding  cost  of  increased  replications.  Based  on  Equation  1  above,  higher 
confidence  levels  or  smaller  acceptable  absolute  errors  result  in  a  greater  number 
replications  required  in  the  experiment.  In  this  experiment,  a  decrease  in  /?  to  1 .0 
vehicles,  requires  approximately  doubling  the  number  of  replications  of  each  design 
point.  As  is  shown  in  the  following  chapter,  the  number  of  replications  required  for  the 
sensitivity  analysis  and  final  experiment  are  12  and  9  respectively;  these  seem 
appropriate  for  this  investigation. 
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DATA  ANALYSIS 


Sensitivity  Analysis 


The  selection  of  the  factor  levels  is  an  arbitrary  decision  based  on  the  composition  of 
the  military  units  in  which  each  weapon  system  is  found.  High  levels  for  the  ADA 
weapons  correspond  to  the  number  of  systems  in  two  ADA  platoons.  Likewise,  the  40 
aircraft,  that  constitute  2  attack  helicopter  squadrons,  are  the  basis  for  the  high  factor 
level  of  the  HIPs  and  HINDs. 

In  order  to  determine  if  the  selected  factor  levels  are  adequate,  this  study  incorporates 
a  sensitivity  analysis.  The  sensitivity  analysis  examines  data  collected  from  the 
experiment  conducted  at  the  initial  factor  level  settings  using  coded  values.  The  result  of 
the  “specified  precision”  technique  determines  the  number  of  required  replications  for 
each  data  point.  For  the  sensitivity  analysis,  two  randomly  selected  points  are  subjected 
to  the  procedure:  data  points  14  and  16.  Based  on  5  initial  replications  of  the  two  data 
points,  a  total  of  12  replications  of  each  point  appears  sufficient  to  provide  an  adequate 
estimate  of  the  remaining  ADA  systems  and  the  remaining  vehicles  in  the  task  force.  See 
Appendix  C  for  a  detailed  explanation  of  the  calculations  for  the  required  replications. 
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Appendix  D  contains  a  complete  listing  of  the  responses  of  each  replication,  for  all  data 
points  in  the  sensitivity  analysis  experiment. 

Utilizing  coded  factor  levels,  a  factorial  analysis  is  conducted  on  the  data  to 
determine  the  effects  of  each  main  factor  on  the  response  variables:  the  amount  of  ADA 
remaining  and  the  amount  of  the  task  force  remaining.  With  a  =  0.05,  the  analysis  shows 
that  the  number  of  Linebacker  weapon  systems  does  not  significantly  contribute  to 
determining  the  number  of  remaining  ADA  systems.  Likewise,  the  number  of  Stingers 
does  not  have  a  significant  effect  on  the  amount  of  the  task  force  remaining.  It  is  also 
apparent  that  the  effects  of  the  HIPs  and  HINDs  are  much  larger  than  that  of  any  other 
factor,  possibly  overwhelming  the  contributions  of  the  other  factors.  Appendix  E 
contains  the  tabular  factorial  analyses  for  the  amount  of  ADA  remaining  and  the  amount 
of  the  task  force  remaining. 

To  address  the  issues  of  main  factor  significance  and  factor  dominance,  the  factor 
levels  are  adjusted  for  use  in  the  final  experiment.  Because  the  HIPs  and  HINDs  appear 
to  be  the  dominant  factors,  their  high  levels  are  adjusted  to  correspond  to  one  squadron  of 
attack  helicopters  each.  It  must  be  understood  that  with  the  adjusted  settings,  the 
experimental  region  is  significantly  smaller  than  before.  Accordingly,  this  restricts  the 
applicability  of  the  results  found  here  to  the  same  region.  Any  extrapolation  of  the 
experimental  results  outside  the  design  space  is  subject  to  considerable  error  and  must  be 
approached  with  caution.  Table  4  contains  the  adjusted  settings  for  all  five  factors. 
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Table  4 

Adjusted  Factor  Levels 

Factor  High  Low 

Stingers  (S)  20  0 

Linebackers  (L)  8  0 

Avengers  (A)  12  0 

HIPs  (Hp)  20  0 

HINDs  (Hd)  20  0 


Final  Experimental  Design 

As  in  the  sensitivity  analysis,  the  method  used  to  determine  the  number  of 
replications  required  in  the  final  experiment  is  the  “specified  precision”.  Once  again  two 
randomly  selected  data  points,  point  7  and  point  10,  are  subjected  to  the  procedure.  Based 
on  5  initial  replications,  a  total  of  9  replications  of  each  point  appears  sufficient  to 
provide  an  adequate  estimate  of  the  responses.  See  Appendix  F  for  a  detailed  explanation 
of  the  calculations  for  the  required  replications.  Appendix  G  contains  a  complete  listing 
of  the  responses  of  each  replication,  for  all  data  points  in  the  final  experiment. 

Using  coded  factor  levels,  a  factorial  analysis  reveals  the  effects  of  all  main  factors 
and  two  factor  interactions  on  the  response  variables.  With  a  =  0.05,  it  is  apparent  that 
with  the  new  factor  levels  all  of  the  main  effects,  as  well  as  many  of  the  interactions,  are 
significant  for  both  responses.  Appendix  H  contains  the  tabular  factorial  analyses  for  the 
percent  of  ADA  remaining  and  the  percent  of  the  task  force  remaining. 
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The  percent  of  the  ADA  remaining  is  calculated  by  dividing  the  number  of  air 
defense  weapon  systems  remaining  at  the  conclusion  of  the  simulation  by  the  initial 
number  of  air  defense  weapons.  It  is  important  to  remember  that  the  initial  number  of  air 
defense  weapons  changes  based  on  the  examined  data  point  from  the  experimental 
design.  Therefore,  the  initial  number  of  air  defense  weapons  is  only  constant  among 
replications  of  the  same  data  point.  The  percent  of  the  task  force  remaining  is  calculated 
by  dividing  the  remaining  task  force  vehicles  by  the  initial  number  of  task  force  vehicles. 
The  initial  number  of  task  force  vehicles  is  53  for  every  replication.  The  task  force 
numbers  do  not  include  any  air  defense  vehicles. 

The  analysis  of  variance  portion  of  the  factorial  analysis  for  both  responses  includes 
a  test  for  curvature.  For  both  models,  the  test  suggests  the  existence  of  curvature  within 
the  response  region,  p  =  0.000  (Appendix  H).  Thus,  a  more  complete  model  for  both 
responses  could  include  the  quadratic  terms  of  each  of  the  five  main  factors.  Ideally, 
axial  points  should  be  added  to  the  existing  design  to  allow  estimation  of  these  effects.  In 
this  experiment,  10  axial  points  could  be  added  to  transform  the  design  into  a  face 
centered  cube  configuration.  It  must  be  understood  that  this  research  utilizes  an 
unclassified  training  database.  The  database  provides  sufficient  accuracy  to  not  mislead 
trainees  as  to  the  performance  of  individual  weapons.  However,  the  database  is  not 
accurate  enough  to  enable  precise  calculation  of  weapon  system  effects.  The  relationships 
between  the  factors  identified  here  are  for  demonstration  purpose  only.  Therefore,  it  is 
assumed  that  the  benefit  gained  from  adding  these  additional  points  to  the  experiment  is 
not  significant  enough  to  warrant  their  inclusion. 
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Regression  analysis  furnishes  a  means  for  determining  a  functional  form  for  each 
response:  the  percentage  of  ADA  systems  remaining  and  the  percentage  of  the  task  force 
remaining.  With  an  objective  of  minimizing  the  mean  square  error  (MSE),  the  regression 
analysis  renders  functional  forms  for  the  relationships  between  the  independent  and 
dependent  variables.  Appendix  I  displays  the  regression  results  for  each  response,  the 
percentage  of  ADA  assets  remaining  and  the  percentage  of  the  task  force  remaining.  In 
order  to  reduce  the  susceptibility  to  round  of  errors,  the  regression  analysis  in  this 
research  was  all  performed  using  coded  values.  The  resulting  relationships  appear  as 
Equations  2  and  3  below. 

%  ADA  remaining  =  0.578  +  0.1 1 1  S  +  0.0872  L  +  0.156  A  -  0.212  Hp  -  0.194 
Hd  -  0.0845  S*L  -  0.0532  S*A  +  0.0263  S*Hp  +  0.0304  S*Hd  -  0.0547  L*A  + 

0.0234  L*Hd  +  0.0350  A*Hp  +  0.0361  A*Hd  +  0.0235  Hp*Hd  (2) 

%  Task  Force  remaining  =  0.698  +  0.0759  S  +  0.0566  L  +  0.0777  A  -  0.182  Hp  - 
0.146  Hd  -  0.0347  S*L  -  0.0291  S*A  +  0.00614  S*Hp  +  0.0235  S*Hd  -  0.0462 
L*A  +  0.0334  L*Hp  -  0.00588  L*Hd  +  0.0529  A*Hp  -  0.00988  A*Hd  -  0.0158 
Hp*Hd  (3) 

The  resulting  R2  values  for  the  regression  equations  are  95.7%  and  94.9%  respectively. 
This  shows  that  most  of  the  variability  in  the  responses  is  explained  by  these  equations. 
The  adjusted  R2  values  are  95.3%  and  94.4%.  The  small  difference  between  the  R2  values 
and  the  adjusted  R2  values  is  an  indication  that  the  model  does  not  contain  unnecessary 
terms. 

Residual  analysis  is  a  useful  tool  for  identifying  problem  areas  with  regression 
models.  Ordinary  residuals  are  the  difference  between  the  fitted  values  of  the  response 
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function  and  the  actual  observed  values.  Under  the  assumptions  of  regression,  errors 
should  follow  a  normal  distribution  (Fox,  1991).  For  the  percentage  of  ADA  and  the  task 
force  remaining,  the  normal  probability  plots  of  the  residuals  do  not  provide  any  reason  to 
doubt  this  assumption  (Appendix  J). 

Typically,  plots  of  the  residuals  against  the  fitted  values  of  the  response  are  used  to 
detect  nonconstant  error  variance  (Cook  &  Weisberg,  1982).  In  this  experiment,  these 
plots  do  not  provide  any  evidence  that  the  assumption  of  constant  error  variance  does  not 
hold  true  in  the  center  of  the  design  space.  However,  at  the  extremes  of  the  responses 
(0%  and  100%),  the  plots  show  there  is  very  little  variance  (Appendix  J).  This  is  a  direct 
result  of  the  definition  of  the  responses.  Logically,  observed  values  of  the  responses 
cannot  be  lower  than  0%  or  greater  than  100%.  Yet,  the  regression  equations  could 
predict  values  that  violate  these  extremes.  The  actual  observed  values  in  these  situations 
would  lie  on  or  near  0%  or  100%.  Thus,  the  error  variance  at  the  extreme  responses  will 
approach  0. 

Residual  plots  can  also  reveal  information  regarding  model  misspecification.  The 
test  for  curvature  previously  indicated  the  possibility  of  curvature  in  the  response  region 
(Cook  &  Weisberg).  Plots  of  the  residuals  against  the  five  factors  of  both  response  also 
support  the  existence  of  curvature  in  the  experimental  region  (Appendix  J).  Due  to  the 
nature  of  the  experimental  design,  it  is  unclear  which  factor  or  factor  requires  a  quadratic 
term.  Clearly,  the  nonlinearity  of  the  main  factors  requires  attention  in  future  research 
efforts. 
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Another  use  of  residual  diagnostics  is  to  assist  in  identifying  points  of  high  influence 
and  outliers.  Observations  with  high  studentized  residuals,  |tj  >  2,  could  be  considered  as 
possible  outliers  (Fox).  Several  observations  of  both  responses  have  observations  with 
high  studentized  residuals  (Appendix  K).  An  additional  measure  to  help  identify  points 
of  influence  is  using  Cook’s  distance  (Cook’s  D).  Observations  with  large  Cook’s  D 
values  are  considered  possible  points  of  high  influence.  Three  observations  with  high 
studentized  residuals  also  have  large  values  for  Cook’s  D  lending  strong  support  to  the 
assertion  that  they  are  leverage  points  (Appendix  K).  It  is  inadvisable  to  remove  the 
leverage  points  found  here  from  this  experiment.  Although  they  influence  the  model, 
they  help  demonstrate  the  inherent  variability  of  a  dynamic  battlefield. 

The  regression  equations  shown  above  clearly  demonstrate  the  existence  of 
interactions  between  the  variables.  However,  the  nature  of  the  interactions  does  not 
support  the  notion  of  the  existence  of  synergy  between  weapon  systems,  a  long  standing 
tenet  of  ADA.  If  synergy  existed,  the  interaction  effects  between  ADA  weapons  would 
be  positive.  For  both  the  ADA  and  task  force  remaining  equations,  note  that  the 
interactions  between  ADA  weapon  systems  are  negative.  In  other  words,  the  net  effect  of 
the  weapons  deployed  in  proximity  with  one  another  is  less  than  if  they  deployed 
separately.  A  possible  explanation  for  this  is  that  it  is  due  to  the  finite  number  of  aircraft 
flown  in  the  experiment.  For  nearly  every  replication  at  every  design  point,  all  of  the 
enemy  aircraft  are  destroyed,  regardless  of  the  amount  of  air  defense  deployed.  With  an 
increase  in  the  amount  of  air  defense  assets,  a  similar  number  of  opportunities  to  destroy 
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aircraft  are  now  shared  among  more  air  defense  systems.  In  an  experiment  designed  with 
a  greater  number  of  aircraft  and  an  intelligent  enemy,  the  nature  of  these  interactions  may 
be  quite  different. 

In  light  of  the  question  of  the  validity  of  the  interaction  terms,  consideration  of  a 
regression  model  in  the  main  effects  only  is  warranted.  Some  argue  that  simplified 
regression  models  are  more  robust  as  they  are  less  dependent  on  the  contributions  of  the 
relationships  between  the  independent  variables.  Equations  4  and  5  below  are  the 
simplified  process  equations  for  the  amount  of  ADA  and  task  force  assets  remaining 
(Appendix  I). 

%ADA  remaining  =  0.578  +  0.1 1 1  S  +  0.0872  L  +  0.156  A  -  0.212  Hp  -  0.194  Hd 

(4) 

%TFremaining  =  0.698  +  0.0759  S  +  0.0566  L  +  0.0777  A  -  0.182  Hp  -  0.146  Hd 

(5) 

The  resulting  regression  produce  R2  values  of  83.7%  (R2  adj.  =  83.1%)  and  83.9% 

(R2  adj.  =  83.4%)  respectively.  This  reveals  that  the  inclusion  of  interaction  terms 
explains  over  1 0%  more  variability  in  the  responses.  An  additional  price  to  pay  for 
process  simplification  is  mean  square  errors  that  approximately  triple  those  associated 
with  the  equations  including  (Appendix  I).  The  lower  R2  values  and  corresponding 
increases  in  MSE,  suggests  that  the  inclusion  of  the  interaction  terms  in  the  model  is 
necessary. 
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With  the  experiment  complete,  the  relationships  between  the  variables  must  be 
integrated  into  the  ADA  planning  process.  Using  the  regression  equations  the  ADA 
planner  will  understand  the  expected  results  of  employing  various  ADA  weapon  system 
packages.  However,  manually  calculating  the  results  of  the  regression  equations  for 
multiple  courses  of  action  is  time  consuming  and  impractical.  In  order  for  them  to  be  be 
useful,  the  regression  equations  must  be  incorporated  into  a  planning  tool  that  reduces  the 
demands  on  the  planner. 
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APPLICATION  AND  LIMITATION 


Application 

Graphical  Approach 

The  development  of  the  regression  equations  for  the  remaining  strength  of  the 
defended  task  force  and  air  defense  weapons  provides  planners  with  the  ability  to 
analytically  approach  the  air  defense  planning  process.  In  a  dynamic  operational 
environment,  the  direct  use  of  the  regression  equations,  to  assist  in  the  force  allocation 
process,  would  be  unwieldy  and  cumbersome  at  best.  However,  techniques  do  exist  for 
determining  acceptable  solutions  for  multiple  response  problems  that  can  be  readily 
applied  in  this  situation. 

In  1959,  Arthur  E.  Hoerl  suggested  a  relatively  simple  technique  of  overlaying 
response  surface  contour  plots  to  determine  regions  of  optimal  response  for  more  than 
one  response  variable.  In  general  terms,  the  analyst  selects  two  of  the  factors  studied  to 
plotted  along  the  X  and  Y  axes.  Holding  all  other  factors  constant,  the  regression 
equation  is  used  to  plot  contours  of  constant  response.  This  process  is  repeated  for  all 
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response  criteria.  The  resulting  contour  plots  overlaid  on  one  another.  From  overlaid 
plots,  the  analyst  can  determine  regions  of  optimality  for  response  variables  studied.  For 
multiple  factors,  the  process  is  repeated  for  all  combinations  of  factors.  The  decision 
maker  uses  all  sets  of  overlays  to  select  the  “best”  point. 

Consider  this  technique  applied  to  the  regressions  equations  developed  for  the 
remaining  strength  of  the  task  force  and  air  defense  assets.  From  the  Intelligence 
Preparation  of  the  Battlefield  (IPB)  developed  for  the  operation  being  planned,  the  air 
defense  planner  has  a  planning  estimate  of  the  amount  and  type  of  enemy  aircraft 
expected.  Provided  the  IPB  estimate  of  the  enemy  aircraft  is  within  the  range  of  the  data 
used  to  develop  the  regression  models,  the  planner  applies  the  graphical  overlay 
technique.  Holding  the  enemy  aircraft  factors  constant  at  the  estimated  levels,  only  three 
factors  remain  for  consideration:  the  numbers  of  Stingers,  Linebackers,  and  Avengers. 
The  planner  can  then  select  one  of  these  weapon  system  to  hold  constant  and  develop 
contour  plots  of  constant  response.  Figures  1  and  2  below  show  contour  plots  of  constant 
response  for  the  percentage  of  the  task  force  remaining  and  the  percentage  of  the  air 
defense  assets  remaining,  respectively.  For  each  contour  plot,  the  expected  enemy  threat 
consists  of  10  HIPs  and  10  HINDs.  Additionally,  the  number  of  Stingers  is  held  constant 
at  10. 
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Figure  1 


Contour  Plot  of  the  Percentage  of  Air  Defense  Assets  Remaining 


Figure  2 


Contour  Plot  of  the  Percentage  of  the  Task  Force  Remaining 
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The  above  plots  can  now  be  overlaid  to  assist  in  determining  an  acceptable  allocation 
of  air  defense  assets.  Assume  for  this  operation  that  60%  air  defense  assets  remaining 
and  70%  of  the  task  force  remaining  is  “acceptable”.  Figure  3  shows  the  combined 
contours  of  constant  response,  depicting  the  “acceptable”  region  as  shaded.  The  planner 
can  now  visually  identify  force  allocation  packages  of  Stingers,  Linebackers,  and 
Avengers  within  the  “acceptable”  region.  It  is  important  to  note  that  this  technique  does 
not  constrain  the  planner  to  one  “best”  solution.  Rather,  it  identifies  an  entire  range  of 
possibilities  that  meet  the  required  constraints. 


Figure  3. 
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Linebackers 


Overlaid  Contour  Plots  of  Remaining  Air  Defense  and  Task  Force  Strength 
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Consider  the  following  example.  A  unit  expects  an  enemy  air  attack  consisting  of  10 
HIPs  and  10  HINDs.  Holding  the  number  of  Stingers  constant  at  10,  a  planner  evaluates 
the  the  combination  of  10  Avengers  and  4  Linebackers  using  Figure  3.  The 
corresponding  response  for  this  combination  of  air  defense  assets  lies  within  the  shaded 
region  of  Figure  3.  Therefore,  this  is  an  “acceptable”  ADA  employment  package. 

The  contour  plots  above  identify  individual  amounts  of  aircraft  and  air  defense 
systems.  Although  the  factor  levels  identified  previously  correspond  to  unit  size 
configurations,  enemy  tactics  and  friendly  force  packages  do  not  always  conform  to  these 
configurations.  Since  HIPs  and  HINDs  attack  in  pairs,  typical  threat  packages  are  any 
combination  of  even  numbers  of  HIPs  and  HINDs.  For  most  operations,  ADA  elements 
fight  as  platoons.  However,  in  some  instances,  platoons  split  into  sections  based  on 
operational  requirements.  Furthermore,  platoons  often  enter  battle  at  less  than  100% 
strength.  For  example,  a  platoon  of  Avengers  may  fight  with  only  five  Avengers  due  to 
maintenance  problems  or  attrition.  This  situation  frequently  occurs  during  subsequent 
battles.  Therefore,  when  selecting  force  allocation  packages,  ADA  commanders  must  not 
consider  squadrons  of  enemy  aircraft  and  platoons  of  air  defense  weapons,  rather  the 
number  of  attacking  aircraft  and  available  ADA  weapons. 
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Prediction  Intervals  and  Spreadsheet  Modeling 


One  shortcoming  of  the  contour  plot  approach  is  that  it  does  not  address  the 
variability  associated  with  each  of  the  responses.  The  plotted  responses  are  mathematical 
estimations  of  their  statistical  mean.  For  each  point  on  the  response  curves,  there  exists  a 
region  of  variability  that  is  not  captured  by  the  graphs.  This  could  lead  to  the  selection  of 
a  force  allocation  package  with  an  unacceptable  amount  of  expected  variability  in  the 
response. 

In  order  to  account  for  variability,  each  predicted  response  of  the  regression  models 
must  be  accompanied  by  a  prediction  interval.  In  the  graphical  approach,  this  would 
involve  plotting  “prediction  bands”  on  either  side  of  the  contours  of  constant  response. 
Adding  two  additional  contour  lines  for  each  response  contour  could  significantly 
increase  the  difficulty  of  analyzing  the  plots.  Representing  the  response  estimates  and 
prediction  intervals  can  be  done  easily  and  clearly  through  computer  based  spreadsheets. 

First,  modeling  the  responses  in  a  spreadsheet  is  a  feasible  solution  for  this  problem. 
In  Microsoft  Excel,  data  tables  provide  a  means  for  determining  the  differences  in  the 
results  of  equations  due  to  changing  up  to  two  variables.  The  data  tables  shown  in  the 
following  discussion  are  the  output  of  a  prototype  force  allocation  planning  tool 
developed  on  MS  Excel.  The  tool  consists  of  a  series  of  linked  spreadsheets.  Each  data 
table  used  above  is  contained  on  a  separate  spreadsheet  in  the  tool.  The  user  specifies  the 
expected  enemy  threat  and  the  number  of  weapons  for  the  ADA  weapon  system  held 
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constant  on  a  separate  data  entry  spreadsheet.  The  tool  dynamically  updates  the 
remaining  spreadsheets  with  each  change  on  the  data  entry  sheet.  No  additional  user 
interaction  is  required  beyond  initial  data  entry.  The  tool’s  ease  of  use  will  advance  its 
acceptance  in  the  field  Army.  Appendix  L  provides  examples  of  each  spreadsheet  in  the 
tool  as  well  as  explains  its  construction. 

For  illustrative  purposes,  consider  the  same  situation  previously  presented  in  the 
contour  plot  discussion.  The  expected  enemy  threat  consists  of  10  HIPs  and  10  HINDs 
and  Stingers  are  chosen  to  remain  constant  at  10  as  well.  Figures  4  and  5  are  sample 
solution  tables. 
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Figure  5 
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As  done  with  contour  plot  method  previously,  assume  that  60%  air  defense  assets 
remaining  and  70%  of  the  task  force  remaining  is  “acceptable”.  As  before,  the  planner 
can  easily  identify  a  range  of  combinations  of  air  defense  assets  that  satisfy  these 
conditions.  For  example,  from  Figures  3  and  4  above,  10  Stingers  (held  constant),  8 
Avengers  and  4  Linebackers  appears  to  be  an  adequate  force  structure  to  defend  against 
10  HIPs  and  10  HINDs  (held  constant).  Likewise,  the  tabular  results  of  the  data  table 
analysis,  in  Figures  4  and  5,  also  indicate  that  this  air  defense  mix  is  “sufficient”,  pointing 
to  an  average  of  63%  air  defense  assets  remaining  and  72%  of  the  task  force  remaining. 
However,  if  the  planner  considers  the  variation  associated  with  this  allocation  of  assets, 
he  may  come  to  a  very  different  conclusion. 
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Secondly,  predictions  intervals  constructed  for  each  point  estimate  of  the  response 
will  provide  the  decision  maker  an  understanding  of  the  variability  associated  with  the 
regression  models.  Prediction  intervals  are  determined  using  Equation  6  below. 


y(x  o )  ±  Wp  (rio-'x  0  (6) 

where: 

MSE  is  the  estimate  of  or2 
n  =  the  number  of  observations  in  the  sample 
p  =  the  number  of  parameters  in  the  model 

[note:  (X'X)~'  for  response  is  contained  in  Appendix  I] 

It  is  important  to  realize  that  the  width  of  the  prediction  interval  is  dependent  on  the 
location  in  the  response  region.  Incorporating  this  formula  into  a  similar  data  to  that  used 
for  the  predicted  response,  will  enable  the  decision  maker  to  more  fully  understand  the 
possible  consequences  of  his  decision.  As  before,  holding  the  number  of  Stingers,  HIPS, 
and  HINDs  constant  at  10  each,  Figures  6  and  7  show  the  half  width  of  the  prediction 
intervals  for  each  response  at  each  combination  of  air  defense  weapons.  The  prediction 
interval  half  widths  are  the  numbers  that  must  be  added  to  and  subtracted  from  the 
predicted  responses  to  form  complete  prediction  intervals,  a  =  0.10.  A  90%  confidence 
interval  is  selected  as  it  seems  a  reasonable  degree  of  precision  for  operational  planning. 
In  practice,  the  confidence  used  should  reflect  the  degree  of  precision  desired  by  the 
commander.  Modeling  in  a  spreadsheet  allows  for  easy  adjustment  of  the  confidence 
level. 
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Figure  6 
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Figure  7 
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For  the  identical  acceptable  conditions  as  before  (at  least  60%  ADA  remaining  and 
at  least  70%  of  the  task  force  remaining),  now  consider  the  same  application  of  the  10 
Stingers,  8  Avengers,  and  4  Linebackers  versus  10  HIPs  and  10  HINDs.  From  Figure  6, 
the  prediction  interval  factor  for  the  percentage  of  ADA  remaining  is  13.6%,  rendering 
the  interval  49.4%  to  86.6%.  Similarly  figure  7  gives  1 1%  as  the  half  width  of  the 
prediction  interval  for  the  percentages  of  the  task  force  remaining  resulting  in  the  interval 
61%  to  83%.  Although  the  predicted  responses  for  the  amount  of  ADA  assets  remaining 
(63%)  and  the  task  force  remaining  (72%)  are  within  permissible  bounds,  the  low  end  of 
prediction  intervals  of  both  responses  extend  well  below  the  “acceptable”  region.  Thus 
considering  the  variability  inherent  in  the  models,  the  decision  maker  may  be  drawn  to  an 
entirely  different  allocation  of  ADA  resources. 

As  indicated  previously,  the  inclusion  of  the  prediction  interval  on  the  graphical 
application  is  possible.  However,  the  additional  of  two  contours  for  each  existing 
contour  on  the  plot  could  greatly  decrease  the  readability  of  the  graphs.  One  possible 
solution  is  to  only  plot  the  contours  of  constant  response  that  correspond  to  the  upper 
ranges  of  the  prediction  intervals.  For  example,  a  point  on  the  contour  at  70%  would  be 
interpreted  as  the  combination  of  variables  whose  mean  response  added  to  the  half  width 
of  the  prediction  interval  sums  to  70%.  The  resulting  graphs  would  be  as  easy  to 
interpret  as  the  plots  illustrated  in  the  figures  in  this  research.  Additional  research  is 
needed  to  determine  if  the  tabular  spreadsheet  presentation  or  the  graphical  presentation 
is  preferable. 
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A  significant  contribution  of  this  research  is  that  it  demonstrates  a  means  to 
analytically  model  the  battlefield.  The  resulting  regression  models  are  readily 
incorporated  into  graphical  and  spreadsheet  applications.  To  illustrate  the  improvement 
that  this  method  makes  in  force  ratio  analysis,  consider  the  same  planning  situation 
analyzed  with  COFA.  COFA  provides  planning  values  for  all  of  the  weapon  systems 
considered  here  except  the  Linebacker.  The  Linebacker  is  variant  of  the  BSFV  which  is 
widely  considered  improvement  in  firepower  and  survivability.  Lacking  an  established 
COFA  value  for  the  Linebacker,  the  BSFV  COFA  value  will  be  used  here  as  a 
conservative  estimate.  Table  5  displays  the  COFA  analysis  for  10  Stingers,  4 
Linebackers,  and  8  Avengers  opposing  10  HIPs  and  10  HTNDs. 


Table  5 

COFA  Analysis  of  Example  Situation 


Type 

Friendly 

Value 

Num. 

Total 

Enemy 

Type  Value 

Num. 

Total 

Stinger 

0.5 

10 

5 

MI-8  HEP  1.8 

10 

18 

Linebacker 

0.6 

4 

2.4 

MI-24  HIND  2.4 

10 

24 

Avenger 

1.9 

8 

15.2 

TOTAL 

22.6 

42 

Force  ratio 

22.6  :42  or  1.0  :  1.85 

Based  on  the  1 :3  force  ratio  convention,  COFA  plainly  indicates  that  the  ADA  assets 
allocated  are  more  than  enough  to  combat  the  expected  enemy  air  threat.  Moreover, 
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COFA  suggests  that  the  Avengers  and  Stingers  are  more  than  enough  air  defense  without 
any  contribution  from  the  Linebackers.  Without  the  Linebackers  and  three  fewer 
Avengers,  the  resulting  force  ratio  of  1:2.9  is  still  better  than  the  1:3  COFA  guideline. 
From  Figures  4  and  5  above  for  this  level  of  air  defense,  the  predicted  amount  of  air 
defense  remaining  is  46%  with  62%  of  the  task  force  remaining.  Most  likely,  few 
commanders  would  accept  predicted  losses  of  this  magnitude.  Even  without  considering 
any  prediction  intervals,  both  of  these  predicted  values  fall  well  below  the  previously 
established  thresholds  of  60%  for  the  ADA  and  70%  for  the  task  force.  One  could  argue 
that  the  60%  and  70%  thresholds  are  arbitrary  and  could  be  adjusted.  However,  COFA 
provides  no  measures  of  performance  to  adjust  the  values.  COFA  merely  offers  the  1 :3 
ratio  as  a  planning  guide  with  only  a  binary  indication  of  the  expected  outcome. 

Limitations 


Unfortunately,  this  research,  as  well  as  the  resulting  planning  tool,  suffers  from 
limitations  that  preclude  its  immediate  introduction  as  a  planning  aid.  Many  of  the 
limitations  are  a  result  of  the  initial  scope  of  the  research.  The  disposition,  mission,  and 
composition  of  the  task  force  in  the  experiment  impose  severe  limitations.  The  scenario 
arrays  a  battalion  task  force  in  a  defensive  posture,  protected  by  various  combinations  of 
air  defense  assets.  All  task  force  and  air  defense  assets  are  stationary  and  occupy 
positions  in  partial  defilade.  Since  defilade  provides  a  measure  of  protection  from 
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observation  and  enemy  fire,  this  research  provides  no  insight  into  the  differences  that 
may  be  encountered  by  units  in  full  defilade.  Likewise,  the  increased  exposure  and 
vulnerability,  characteristic  of  offensive  operations,  are  not  addressed.  Additionally,  the 
task  force  composition  and  activities  remained  static  throughout  the  simulation.  Clearly, 
differing  types  and  amounts  of  tanks  and  Infantry  Fighting  Vehicles  would  influence  the 
battle  as  did  differing  types  and  amounts  of  air  defense  vehicles.  This  work  does  not 
capture  these  influences. 

Limitations  also  exist  on  the  enemy  side.  The  task  force  is  attacked  by  various 
configurations  of  enemy  aircraft,  however,  no  enemy  ground  threat  accompanies  the  air 
attack.  The  effect  an  enemy  ground  attack  on  the  air  defense  and  task  force  remaining  as 
well  as  any  interaction  between  enemy  ground  forces  and  air  attack  remain  unknown. 
Additionally,  ,  the  task  force  does  not  fight  an  “intelligent  or  thinking”  enemy.  All  of  the 
enemy  attack  routes  are  preplanned.  This  results  in  an  enemy  that  attacks  relentlessly, 
without  regard  to  its  losses  or  successes  on  the  battlefield.  It  is  highly  likely,  that  this 
artificiality  introduces  a  disproportionate  number  of  friendly  and  enemy  losses  when 
compared  to  battles  conducted  with  knowledgeable  opponents. 

In  the  experiment,  the  effects  of  terrain,  time  of  day,  and  weather  are  controlled. 

The  mountainous  desert  terrain,  daylight  hours,  and  fair  weather  encountered  in  the 
simulation  favor  all  play  an  influential  role  in  determining  visibility  and  trafficability.  A 
different  combination  of  terrain,  time,  and  weather  could  drastically  impact  the  results 


62 


obtained  here.  Applying  the  relationships  established  here  to  situations  with  different 
environmental  conditions  must  be  approached  with  extreme  caution. 

The  experimental  factor  levels  selected  for  the  friendly  and  enemy  units  constrain 
the  study  further.  The  relationships  established  between  the  variables  can  only  be 
considered  valid  within  the  response  region.  Thus,  when  utilizing  the  developed 
regression  equations,  an  analyst  must  insure  that  the  number  of  aircraft  and  air  defense 
systems  do  not  exceed  the  high  factor  levels  used  in  the  experiment.  Using  values,  for 
any  of  the  air  defense  weapons  or  the  enemy  aircraft  greater  than  high  factor  levels, 
could  result  in  significant  error. 

A  final  shortcoming  of  this  work  is  that  the  Janus  suite  used  incorporates  an 
unclassified  “training”  database.  The  U.S.  Army  considers  actual  performance  data  for 
many  weapon  systems  to  be  “classified”.  For  unit  level  training,  the  Army  typically 
utilizes  “unclassified”  weapons  system  performance  data  for  training  use,  reserving  the 
use  of  “classified”  data  for  detailed  analysis  and  operational  planning.  For  academic  and 
publishing  considerations,  “unclassified”  data  were  used  here. 
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CONCLUSION  AND  RECOMMENDATIONS 


The  use  of  force  ratios  is  an  extremely  valuable  planning  tool  for  the  U.S.  Army. 
Analysis  of  force  ratios  provides  planning  staffs  and  commanders  an  indication  of  the 
required  combat  power  for  successful  operations.  While  the  use  of  force  ratios  is 
prevalent  for  analyzing  maneuver  force  allocation,  little  effort  has  been  made  to  apply 
this  technique  to  air  defense  operations.  The  initial  attempt  at  force  ratio  analysis  within 
the  air  defense  domain,  Correlation  of  Forces  Air,  achieves  some  success,  however,  the 
process  suffers  from  several  shortcomings.  These  shortcomings  include:  the  questionable 
validity  of  the  relative  combat  power  values,  the  exclusion  of  the  possible  synergistic 
effect  of  employing  a  mix  of  diverse  weapon  systems  together,  the  failure  to  account  for 
the  differing  dynamics  of  offensive  and  defensive  operations,  and  the  inadequate  address 
of  success  criteria.  With  the  increase  in  the  number  of  unit  deployments,  the  need  arises 
for  a  development  of  an  air  defense  force  ratio  planning  tool  that  addresses  the  short  falls 
of  Correlation  of  Forces  Air,  COFA.  This  study  addresses  two  of  these  shortcomings: 
the  consideration  of  synergistic  effects  of  weapon  systems  and  success  criteria. 

This  research  develops  a  methodology  that  is  a  substantial  improvement  over  COFA 
for  air  defense  planning.  Through  the  use  of  a  designed  experiment,  the  study 
unquestionably  demonstrates  the  possibility  of  modeling  the  modern  air  defense  battle. 
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Using  Janus,  an  established  military  training  simulation,  a  battlefield  scenario  simulates 
a  representative  air  defense  battle.  Multiple  replications  of  the  simulation  at  numerous 
design  points  fixed  the  relationships  existing  among  the  air  defense  and  air  threat  weapon 
systems.  The  experiment  produced  two  distinct  response  equations  modeling  the  effects 
of  the  various  weapon  systems  on  determining  the  remaining  air  defense  and  task  force 
assets.  Unlike  with  traditional  force  ratio  techniques,  now  the  decision  maker  has 
quantifiable  measures  of  effectiveness  for  air  defense  operations. 

The  development  of  the  means  to  assess  these  two  success  criteria  provides  a  solid 
foundation  for  the  construction  of  alternative  force  allocation  planning  tools.  Using  a 
graphical  technique  of  comparing  contours  of  constant  response,  an  operational  planner 
can  identify  a  range  of  feasible  force  allocation  courses  of  action  for  air  defense  units,  at 
a  glance.  For  each  course  of  action,  the  decision  maker  will  understand  the  expected 
values  of  both  success  criteria:  the  percentage  of  ADA  and  the  task  force  remaining. 

Although  an  attractive  solution,  the  simple  graphical  approach  fails  to  consider  the 
inherent  variability  associated  with  each  of  the  responses.  On  the  other  hand,  examining 
the  predicted  responses  and  associated  variability  of  regression  equations  readily  lends 
itself  to  spreadsheet  data  table  analysis.  Developed  for  this  research,  a  spreadsheet  based 
air  defense  force  allocation  planning  tool  rapidly  produces  point  estimates  of  the 
responses  for  multiple  combinations  of  air  defense  assets  opposing  an  established  air 
threat.  Additionally,  the  tool  generates  the  data  needed  to  quickly  construct  prediction 
intervals  for  each  predicted  response.  The  spreadsheet  planning  tool  equips  the  ADA 
commander  with  the  information  required  to  make  crucial  force  allocation  decisions. 
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Moreover,  the  tool’s  simple  design  and  ease  of  use  enable  it  to  be  utilized  with  minimal 
instruction. 

While  this  research  has  resulted  in  a  new  methodology  for  force  ratio  analysis  for  the 
air  defense  battle,  the  research  is  limited  in  several  ways.  The  scope  of  the  friendly  and 
enemy  missions,  force  compositions,  and  the  constant  environmental  conditions  limit  the 
applicability  of  the  findings.  Extrapolations  of  the  results  found  here  to  situations 
outside  the  experimental  region  are  subject  to  considerable  error.  Further,  since  the 
scenario  did  not  include  an  enemy  ground  force,  it  is  not  prudent  to  assume  that  the 
relationships  found  in  this  experiment  apply  when  a  ground  threat  is  present. 

Additionally  intelligent  interaction  did  not  exist  on  either  the  enemy  or  friendly  side. 
Thus,  combatants  were  restricted  to  preset  plans  that  do  not  account  for  the  adjustments 
that  intelligent  agents  engaged  battle  might  make.  Finally,  it  must  be  understood  that  the 
unclassified  database  used  provides  valuable  insight  into  the  battlefield.  However,  it  is 
not  sufficiently  accurate  for  establishing  precise  analytical  relationships  between  the 
variables. 

As  stated  previously,  the  most  significant  contribution  of  this  research  is  that  it 
provides  a  new  methodology  to  analytically  model  the  air  defense  battlefield.  Using  this 
methodology  as  a  framework,  it  is  recommended  that  expanded  experiments  are 
conducted  that  account  for  this  research’s  limitations.  Separate  regression  models  could 
be  developed  for  different  unit  missions,  environmental  conditions,  and  enemy  tactics. 
While  difficult  to  control,  it  is  critical  to  include  human  interaction  into  the  experiment. 
The  addition  of  intelligent  combatants  would  significantly  heighten  the  realism  of  the 
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scenarios  and  help  align  the  results  of  the  experiment  with  reality.  The  utilization  of  a 
classified  database  would  ensure  that  the  variable  relationships  obtained  would  be  as 
close  to  valid  as  possible. 

Once  the  variable  relationships  from  the  expanded  experiments  are  obtained,  they 
must  be  incorporated  into  planning  tool.  In  this  research,  both  graphical  and  tabular 
approaches  are  explored.  Additional  research  is  required  to  determine  which  technique  is 
best  suited  for  development  into  an  integrated  air  defense  force  ratio  planning  tool. 

The  significance  of  this  research  is  not  limited  to  the  domain  of  air  defense  artillery. 
Like  COFA,  the  force  ratio  process  for  maneuver  forces  does  not  clearly  define  measures 
of  effectiveness  or  success  criteria.  Utilizing  the  same  methodology  applied  here, 
maneuver  force  planning  could  be  equally  improved.  Expanded  experiments  considering 
the  range  of  factors  influencing  the  battle  could  certainly  produce  analytical  models  of 
the  maneuver  battlefield.  As  with  the  models  developed  for  the  air  defense  battle,  the 
maneuver  models  would  provide  the  basis  for  force  allocation  planning  tools  that  provide 
commanders  a  clear  understanding  of  the  consequences  of  their  decisions. 
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Appendix  A 
Glossary 
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ADA  -  Air  Defense  Artillery. 


AT6  Spiral  -  Anti-tank  missile  produced  by  the  former  Soviet  Union.  It  has  a 
maximum  range  of  approximately  6000  m  (HQ  DA,  1991). 

AT  bn  -  Anti  Tank  Battalion. 

ATGM  -  Anti-tank  guided  missile. 

Atk  hel  sqdn  -  Attack  helicopter  squadron. 

Avenger  -  A  U  S.  Army  air  defense  weapon.  Consists  of  a  turret  mounted  on  the  bed  of 

a  HMMWV  containing  8  ready  to  fire  Stinger  missiles  (USAADASCH,  1991). 

BFV  -  Bradley  Fighting  Vehicle  -  M2.  A  U.S.  Army,  light  armored,  infantry  vehicle 
capable  of  carrying  six  soldiers  in  addition  to  a  three  man  crew.  The  BFV  is  armed 
with  a  25  mm  chain  gun,  a  7.62  mm  coaxial  machine  gun,  and  a  TOW  missile 
launcher  (Janes  Armour  and  Artillery  [Janes],  1992). 

BMP -2  -  A  lightly  armored  variant  of  the  BMP-1  infantry  combat  vehicle  produced  by 
the  former  Soviet  Union.  It’s  armament  includes  a  30  mm  main  gun,  a  7.62  mm 
coaxial  machine  gun,  and  an  anti-tank  missile  launcher  (HQ  DA,  1991). 
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Bn  -  battalion. 


BSFV  -  Bradley  Stinger  Fighting  Vehicle.  A  U.S.  Army  air  defense  vehicle  based  on  the 
BFV  (HQ  DA,  95).  The  newest  version  is  called  the  Linebacker  (USAADASCH, 
1996). 

BTR  -  A  lightly  armored  wheeled  personnel  carrier  manufactured  by  the  former  Soviet 
Union.  Several  variants  exist  with  the  most  common  armament  including  a  14.5mm 
machine  gun  and  a  7.62mm  coaxial  machine  gun  (HQ  DA,  1991). 

Cav  sqdn  -  Cavalry  squadron. 

COFA  -  Correlation  of  Forces  Air. 

Div  recon  -  Division  reconnaissance  element. 

HMMWV  -  High  Mobility  Military  Wheeled  Vehicle. 

Linebacker  -  The  latest  version  of  the  BSFV.  Replaces  the  TOW  missile  launcher  with  a 
Stinger  missile  pod  with  four  ready  to  fire  missiles  (USAADASCH,  1996). 
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Ml  A1  -  The  current  U.S.  Army  main  battle  tank  named  the  Abrams.  The  tank  is  armed 
with  a  120  mm  smooth  bore  gun,  a  7.62  mm  coaxial  machine  gun,  and  a  12.7  mm 
machine  gun.  The  tank  is  an  improvement  over  the  M60  series  tanks  in  protection, 
mobility,  and  firepower  (Janes,  1992). 

M60  -  A  U.S.  Army  main  battle  tank..  The  tank  is  armed  with  a  105  mm  rifled  gun,  a 
7.62  mm  coaxial  machine  gun,  and  a  12.7  mm  machine  gun.  The  tank  has  been 
almost  entirely  replaced  in  the  active  army  by  the  Ml  Abrams  tank  (Janes,  1992). 

Mi-8  HIP  -  A  troop  carrying  helicopter  manufactured  by  the  former  Soviet  Union  and 
exported  to  its  allies.  The  helicopter  is  available  with  several  weapon  system 
configurations  that  include  a  variety  of  rockets,  bombs,  anti-tank  guided  missiles  and 
machine  guns  (HQ  DA,  1991). 

Mi-24  HIND  -  An  attack  helicopter,  also  capable  of  troop  transport,  manufactured  by  the 
former  Soviet  Union  and  exported  to  its  allies.  Dubbed  the  “flying  tank”,  the 
helicopter  is  heavily  armored  and  is  available  with  several  weapon  system 
configurations  that  include  a  variety  of  rockets,  bombs,  anti-tank  guided  missiles, 
and  machine  guns  (HQ  DA,  1991). 

OPTEMPO  -  Operational  Tempo 


71 


Stinger  -  An  U.S.  Army,  shoulder  fired,  short  range,  air  defense  missile  (HQ  DA,  1985). 
The  missile  is  also  incorporated  in  other  weapon  systems  such  as  the  BSFV  and 
Avenger. 


TOW  -  Tube  lauched,  Optically  tracked  Wire  guided  missile.  A  U.S.  Army  wire  guided 
anti-tank  missile  (HQ  DA,  1995). 


USACGSC  -  The  United  States  Army  Command  and  General  Staff  School. 
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Appendix  B 
Experimental  Design 
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The  following  table  depicts  the  experimental  design  in  coded  variable  form. 


Run 

S 

L 

A 

Hp 

Hd 

Remarks 

1 

-1 

-1 

-1 

-1 

1 

2 

1 

-1 

-1 

-1 

-1 

no  run 

3 

-1 

1 

-1 

-1 

-1 

no  run 

4 

1 

1 

-1 

-1 

1 

5 

-1 

-1 

1 

-1 

-1 

no  run 

6 

1 

'  -1 

1 

-1 

1 

7 

-1 

1 

1 

-1 

1 

8 

1 

1 

1 

-1 

-1 

no  run 

9 

-1 

-1 

-1 

1 

-1 

10 

1 

-1 

-1 

1 

1 

11 

-1 

1 

-1 

1 

1 

12 

1 

1 

-1 

1 

-1 

13 

-1 

-1 

1 

1 

1 

14 

1 

-1 

1 

1 

-1 

15 

-1 

1 

1 

1 

-1 

16 

1 

1 

1 

1 

1 

Center 

0 

0 

0 

0 

0 

(Data  points  with  the  remark  “no  run”  indicate  that  no  simulation  run  was  required  due  to 
the  lack  of  enemy  aircraft.  The  following  page  shows  the  design  in  natural  variable 
form.) 

Resolution  V,  1/2  fraction. 


Design  Generators: 

E  =  ABCD 

Defining  Relation: 

I  =  ABCDE 

Alias  Structure 

A  +  BCDE 

AB  +  CDE 

BD  +  ACE 

B  +  ACDE 

AC  +  BDE 

BE  +  ACD 

C  +  ABDE 

AD  +  BCE 

CD  +  ABE 

D  +  ABCE 

AE  +  BCD 

CE  +  ABD 

E  +  ABCD 

BC  + ADE 

DE  + ABC 
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The  following  table  depicts  the  experimental  design  in  coded  variable  form. 


Run 

S 

L 

A 

Hp 

Hd 

Remarks 

1 

0 

0 

0 

2 

20 

0 

0 

0 

0 

no  run 

3 

0 

8 

0 

0 

0 

no  run 

4 

20 

8 

0 

0 

20 

5 

0 

0 

12 

0 

0 

no  run 

6 

20 

0 

12 

0 

20 

7 

0 

8 

12 

0 

20 

8 

20 

8 

12 

0 

0 

no  run 

9 

0 

0 

0 

20 

0 

10 

20 

0 

0 

20 

20 

11 

0 

8 

0 

20 

20 

12 

20 

8 

0 

20 

0 

13 

0 

0 

12 

20 

20 

14 

20 

0 

12 

20 

0 

15 

0 

8 

12 

20 

0 

16 

20 

8 

12 

20 

20 

Center 

10 

4 

6 

10 

10 

(Data  points  with  the  remark  “no  run”  indicate  that  no  simulation  run  was  required  due  to 
the  lack  of  enemy  aircraft.) 


Appendix  C 

Replication  Calculation  for  the  Sensitivity  Analysis 
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To  determine  the  number  of  required  replications,  the  observed  values  from  the  initial  5 
replications  of  two  randomly  selected  data  points,  are  used  to  calculate  an  estimate  of  the 
variance. 


Data  Point  14 


Data  Point  16 


Systems  Remaining 
Replicate  ADA  Task  Force 


1 

21 

34 

2 

17 

27 

3 

16 

25 

4 

19 

28 

5 

12 

25 

Variance 

11.5 

13.7 

Systems  Remaining 
Replicate  ADA  Task  Force 


1 

38 

42 

2 

33 

41 

3 

40 

41 

4 

40 

39 

5 

40 

40 

Variance 

9.2 

1.3 

Equation  1,  listed  below,  utilizes  the  calculated  variances  to  estimate  the  required  number 
of  replications  («*).  For  this  experiment  P  =1.5  systems  and  a  =  0.02. 

na  =  min  >  n.  -/ S2  («)  /  7  <  p 

(1) 

where: 

n  =  number  of  replications  in  the  sample 
i  =  projected  number  of  replications 

S ?(n)  =  estimate  of  population  variance  based  on  the  sample 
P  =  |  X  -  p|  =  absolute  error  of  the  estimate. 

Data  Point  14  Data  Point  16 


ADA  Task  Force 


1 

error 

error 

5 

2.3249 

2.5376 

6 

2.0434 

2.2303 

7 

1.8457 

2.0145 

8 

1.6965 

1.8517 

9 

1.5792 

1.7236 

10 

1.4831 

1.6188 

11 

1.4028 

1.5312 

12 

1.3343 

1.4563 

ADA  Task  Force 


1 

error 

error 

5 

2.0795 

0.7817 

6 

1.8277 

0.6870 

7 

1.6508 

0.6206 

8 

1.5174 

0.5704 

9 

1.4124 

0.5309 

10 

1.3265 

0.4986 

11 

1.2547 

0.4717 

12 

1.1934 

0.4486 

At  12  replications,  it  is  projected  that  the  errors  on  the  estimates  are  all  below  P  -  1.5 
systems. 
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Appendix  D 

Sensitivity  Analysis  Response  Data 
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Point  1 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

0.0% 

41 

21.2% 

2 

0 

0.0% 

45 

13.5% 

3 

0 

0.0% 

43 

17.3% 

4 

0 

0.0% 

43 

17.3% 

5 

0 

0.0% 

41 

21.2% 

6 

0 

0.0% 

43 

17.3% 

7 

0 

0.0% 

40 

23.1% 

8 

0 

0.0% 

44 

15.4% 

9 

0 

0.0% 

41 

21.2% 

10 

0 

0.0% 

44 

15.4% 

11 

0 

0.0% 

40 

23.1% 

12 

0 

0.0% 

41 

21.2% 

Point  2  (No  aircraft  in 

Run  Destroyed 

experimental  design  - 

ADA 

Remaining 

no  simulation  runs  performed.) 

Task  Force 

Destroyed  Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

10 

0 

100.0% 

0 

100.0% 

11 

0 

100.0% 

0 

100.0% 

12 

0 

100.0% 

0 

100.0% 
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Point  3  (No  aircraft  in  experimental  design  - 

no  simulation  runs  performed.) 

Run 

Destroyed 

ADA 

Remaining 

Task  Force 

Destroyed  Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

10 

0 

100.0% 

0 

100.0% 

11 

0 

100.0% 

0 

100.0% 

12 

0 

100.0% 

0 

100.0% 

Point  4 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

18 

35.7% 

21 

59.6% 

2 

20 

28.6% 

34 

34.6% 

3 

21 

25.0% 

29 

44.2% 

4 

18 

35.7% 

22 

57.7% 

5 

25 

10.7% 

37 

28.8% 

6 

20 

28.6% 

24 

53.8% 

7 

18 

35.7% 

29 

44.2% 

8 

22 

21.4% 

36 

30.8% 

9 

22 

21.4% 

27 

48.1% 

10 

21 

25.0% 

24 

53.8% 

11 

19 

32.1% 

20 

61.5% 

12 

20 

28.6% 

33 

36.5% 
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Point  5  (No  aircraft  in  experimental  design  -  no  simulation  runs  performed.) 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

10 

0 

100.0% 

0 

100.0% 

11 

0 

100.0% 

0 

100.0% 

12 

0 

100.0% 

0 

100.0% 

Point  6 

ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

8 

75.0% 

16 

69.2% 

2 

19 

40.6% 

25 

51.9% 

3 

9 

71.9% 

17 

67.3% 

4 

7 

78.1% 

13 

75.0% 

5 

8 

75.0% 

15 

71.2% 

6 

16 

50.0% 

28 

46.2% 

7 

9 

71.9% 

18 

65.4% 

8 

9 

71.9% 

16 

69.2% 

9 

9 

71.9% 

13 

75.0% 

10 

6 

81.3% 

22 

57.7% 

11 

9 

71.9% 

32 

38.5% 

12 

14 

56.3% 

19 

63.5% 
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Point  7 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

18 

10.0% 

29 

44.2% 

2 

7 

65.0% 

13 

75.0% 

3 

11 

45.0% 

20 

61.5% 

4 

6 

70.0% 

15 

71.2% 

5 

5 

75.0% 

19 

63.5% 

6 

14 

30.0% 

21 

59.6% 

7 

5 

75.0% 

14 

73.1% 

8 

10 

50.0% 

17 

67.3% 

9 

12 

40.0% 

13 

75.0% 

10 

8 

60.0% 

13 

75.0% 

11 

5 

75.0% 

9 

82.7% 

12 

12 

40.0% 

13 

75.0% 

Point  8  (No  aircraft  in  experimental  design 

ADA 

Run  Destroyed  Remaining 

-  no  simulation  runs  performed.) 

Task  Force 

Destroyed  Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

10 

0 

100.0% 

0 

100.0% 

11 

0 

100.0% 

0 

100.0% 

12 

0 

100.0% 

0 

100.0% 
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Point  9 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

0.0% 

39 

25.0% 

2 

0 

0.0% 

39 

25.0% 

3 

0 

0.0% 

39 

25.0% 

4 

0 

0.0% 

39 

25.0% 

5 

0 

0.0% 

39 

25.0% 

6 

0 

0.0% 

39 

25.0% 

7 

0 

0.0% 

39 

25.0% 

8 

0 

0.0% 

39 

25.0% 

9 

0 

0.0% 

39 

25.0% 

10 

0 

0.0% 

39 

25.0% 

11 

0 

0.0% 

39 

25.0% 

12 

0 

0.0% 

39 

25.0% 

Point  10 

Run 

ADA 

Destroyed  Remaining 

Task  Force 

Destroyed  Remaining 

1 

20 

0.0% 

43 

17.3% 

2 

20 

0.0% 

42 

19.2% 

3 

20 

0.0% 

45 

13.5% 

4 

20 

0.0% 

44 

15.4% 

5 

20 

0.0% 

48 

7.7% 

6 

20 

0.0% 

49 

5.8% 

7 

20 

0.0% 

50 

3.8% 

8 

20 

0.0% 

44 

15.4% 

9 

20 

0.0% 

46 

11.5% 

10 

20 

0.0% 

46 

11.5% 

11 

20 

0.0% 

48 

7.7% 

12 

20 

0.0% 

45 

13.5% 
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Point  1 1 


ADA  Task  Force 

Run  Destroyed  Remaining _ Destroyed  Remaining 


1 

8 

0.0% 

46 

11.5% 

2 

8 

0.0% 

47 

9.6% 

3 

8 

0.0% 

47 

9.6% 

4 

8 

0.0% 

44 

15.4% 

5 

8 

0.0% 

49 

5.8% 

6 

8 

0.0% 

50 

3.8% 

7 

8 

0.0% 

43 

17.3% 

8 

8 

0.0% 

47 

9.6% 

9 

8 

0.0% 

45 

13.5% 

10 

8 

0.0% 

48 

7.7% 

11 

8 

0.0% 

45 

13.5% 

12 

8 

0.0% 

45 

13.5% 

Point  12 

Run 

ADA 

Destroyed  Remainins 

Task  Force 

Destroyed  Remainins 

1 

28 

0.0% 

38 

26.9% 

2 

26 

7.1% 

39 

25.0% 

3 

28 

0.0% 

37 

28.8% 

4 

26 

7.1% 

37 

28.8% 

5 

26 

7.1% 

36 

30.8% 

6 

27 

3.6% 

39 

25.0% 

7 

28 

0.0% 

39 

25.0% 

8 

28 

0.0% 

37 

28.8% 

9 

26 

7.1% 

39 

25.0% 

10 

28 

0.0% 

39 

25.0% 

11 

28 

0.0% 

39 

25.0% 

12 

28 

0.0% 

39 

25.0% 
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Point  13 


ADA  Task  Force 

Run  Destroyed  Remaining _ Destroyed  Remaining 


1 

12 

0.0% 

42 

19.2% 

2 

12 

0.0% 

45 

13.5% 

3 

12 

0.0% 

43 

17.3% 

4 

12 

0.0% 

43 

17.3% 

5 

12 

0.0% 

41 

21.2% 

6 

12 

0.0% 

39 

25.0% 

7 

12 

0.0% 

39 

25.0% 

8 

12 

0.0% 

43 

17.3% 

9 

12 

0.0% 

45 

13.5% 

10 

12 

0.0% 

42 

19.2% 

11 

12 

0.0% 

42 

19.2% 

12 

12 

0.0% 

42 

19.2% 

Point  14 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

21 

34.4% 

34 

34.6% 

2 

17 

46.9% 

27 

48.1% 

3 

16 

50.0% 

25 

51.9% 

4 

19 

40.6% 

28 

46.2% 

5 

12 

62.5% 

25 

51.9% 

6 

13 

59.4% 

28 

46.2% 

7 

11 

65.6% 

26 

50.0% 

8 

19 

40.6% 

35 

32.7% 

9 

13 

59.4% 

25 

51.9% 

10 

11 

65.6% 

25 

51.9% 

11 

21 

34.4% 

29 

44.2% 

12 

22 

31.3% 

29 

44.2% 
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Point  15 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

15 

25.0% 

30 

42.3% 

2 

18 

10.0% 

33 

36.5% 

3 

9 

55.0% 

21 

59.6% 

4 

8 

60.0% 

19 

63.5% 

5 

19 

5.0% 

34 

34.6% 

6 

15 

25.0% 

25 

51.9% 

7 

13 

35.0% 

18 

65.4% 

8 

18 

10.0% 

37 

28.8% 

9 

20 

0.0% 

35 

32.7% 

10 

13 

35.0% 

28 

46.2% 

11 

12 

40.0% 

31 

40.4% 

12 

13 

35.0% 

26 

50.0% 

Point  16 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

38 

5.0% 

42 

19.2% 

2 

33 

17.5% 

41 

21.2% 

3 

40 

0.0% 

41 

21.2% 

4 

40 

0.0% 

39 

25.0% 

5 

40 

0.0% 

40 

23.1% 

6 

40 

0.0% 

43 

17.3% 

7 

40 

0.0% 

40 

23.1% 

8 

40 

0.0% 

43 

17.3% 

9 

38 

5.0% 

39 

25.0% 

10 

40 

0.0% 

40 

23.1% 

11 

40 

0.0% 

40 

23.1% 

12 

40 

0.0% 

41 

21.2% 
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Center  Point 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

6 

70.0% 

8 

84.6% 

2 

9 

55.0% 

15 

71.2% 

3 

9 

55.0% 

12 

76.9% 

4 

10 

50.0% 

10 

80.8% 

5 

9 

55.0% 

16 

69.2% 

6 

7 

65.0% 

15 

71.2% 

7 

4 

80.0% 

8 

84.6% 

8 

5 

75.0% 

13 

75.0% 

9 

9 

55.0% 

8 

84.6% 

10 

5 

75.0% 

10 

80.8% 

11 

12 

40.0% 

17 

67.3% 

12 

8 

60.0% 

6 

88.5% 
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Appendix  E 

Estimated  Effects  for  the  Sensitivity  Analysis 
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Estimated  Effects  and  Coefficients  for  ADA  Remaining 


Term 

Effect 

Coef 

Std  Coef 

t-value 

P 

Constant 

0.4068 

0.01309 

31.08 

0.000 

S 

0.0859 

0.0429 

0.01349 

3.18 

0.002 

L 

-0.0050 

-0.0025 

0.01349 

-0.19 

0.853 

A 

0.2128 

0.1064 

0.01349 

7.89 

0.000 

Hp 

-0.5827 

-0.2914 

0.01349 

-21.59 

0.000 

Hd 

-0.4117 

-0.2058 

0.01349 

-15.25 

0.000 

Analysis  of  Variance  for  ADA  Remaining 


Source 

DF 

SeqSS 

Adj  SS 

Adj  MS 

F 

P 

Main  Effects 

5 

26.9578 

26.9578 

5.39157 

154.23 

0.000 

Residual  Error 

198 

6.9217 

6.9217 

0.03496 

Curvature 

1 

0.5393 

0.5393 

0.53929 

16.65 

0.000 

Lack  of  Fit 

10 

4.9002 

4.9002 

0.49002 

61.82 

0.000 

Pure  error 

187 

1.4822 

1.4822 

0.00793 

Total 

203 

33.8795 
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Estimated  Effects  and  Coefficients  for  the  Task  Force  Remaining 


Term 

Effect 

Coef 

Std  Coef 

t-value 

P 

Constant 

0.5183 

0.00097 

53.29 

0.000 

S 

0.0333 

0.0166 

0.01003 

1.66 

0.099 

L 

0.0457 

0.0228 

0.01003 

2.28 

0.024 

A 

0.1554 

0.0777 

0.01003 

7.75 

0.000 

Hp 

-0.4864 

-0.2432 

0.01003 

-24.26 

0.000 

Hd 

-0.3554 

-0.1777 

0.01003 

-17.72 

0.000 

Analysis  of  Variance  for  the  Task  Force  Remaining 

Source 

DF 

SeqSS 

Adj  SS 

Adj  MS 

F 

P 

Main  Effects 

5 

18.7299 

18.7299 

3.74598 

194.12 

0.000 

Residual  Error 

198 

3.8209 

3.8209 

0.0193 

Curvature 

1 

0.8656 

0.8656 

0.8656 

57.7 

0.000 

Lack  of  Fit 

10 

2.199 

2.199 

0.2199 

54.37 

0.000 

Pure  error 

187 

0.7563 

0.7563 

0.00404 

Total 

203 

22.5509 

90 


Appendix  F 

Replication  Calculation  for  the  Final  Experiment 
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To  determine  the  number  of  required  replications,  the  observed  values,  from  the  initial  5 
replications  of  two  randomly  selected  data  points,  are  used  to  calculate  an  estimate  of  the 
variance. 


Data  Point  1 


Data  Point  10 


Replicate 

Systems  Remaining 
ADA  Task  Force 

Replicate 

Systems  Remaining 
ADA  Task  Force 

1 

15 

49 

1 

49 

42 

2 

16 

49 

2 

49 

41 

3 

16 

43 

3 

43 

41 

4 

16 

51 

4 

51 

39 

5 

17 

50 

5 

50 

40 

Variance 

0.5 

9.8 

Variance 

1.8 

1.2 

Equation  1,  listed  below,  utilizes  the  calculated  variances  to  estimate  the  required  number 
of  replications  («* ).  For  this  experiment  (3  =  1.5  systems  and  a  =  0.02. 

na  =  mm^>n:ti_x  x_yJS2(n)li  <  p 

0) 

where: 

n  =  number  of  replications  in  the  sample 

/  =  projected  number  of  replications 

^(n)  =  estimate  of  population  variance  based  on  the  sample 

P  =  |  X  -  (i|  =  absolute  error  of  the  estimate. 

Data  Point  7  Data  Point  10 


ADA  Task  Force  ADA  Task  Force 


1 

error 

error 

1 

error 

error 

5 

0.4848 

2.1462 

5 

0.9198 

0. 7510 

6 

0.4261 

1.8864 

6 

0.8084 

0.6601 

7 

0.3849 

1.7038 

7 

0.7302 

0.5962 

8 

0.3538 

1.5661 

8 

0.6712 

0.5480 

9 

0.3293 

1.4578 

9 

0.6248 

0.5101 

At  9  replications,  it  is  projected  that  the  errors  on  the  estimates  are  all  below  P  =1.5 
systems. 
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Appendix  G 

Final  Experimental  Design  Response  Response  Data 
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Point  1 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

0.0% 

25 

51.9% 

2 

0 

0.0% 

25 

51.9% 

3 

0 

0.0% 

19 

63.5% 

4 

0 

0.0% 

36 

30.8% 

5 

0 

0.0% 

25 

51.9% 

6 

0 

0.0% 

22 

57.7% 

7 

0 

0.0% 

26 

50.0% 

8 

0 

0.0% 

28 

46.2% 

9 

0 

0.0% 

26 

50.0% 

Point  2  (No  aircraft  in 

experimental  design 

-  no  simulation  runs  performed.) 

ADA 

Task  Force 

Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

'  0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 
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Point  3  (No  aircraft  in  experimental  design  -  no  simulation  runs  performed.) 


ADA 


Task  Force 


Run 

Destroyed 

Remainina 

Destroyed 

Remaini 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

Point  4 


ADA  Task  Force 


Run 

Destroyed 

Remainina 

Destroyed 

Remainina 

1 

9 

67.9% 

3 

94.2% 

2 

13 

53.6% 

9 

82.7% 

3 

9 

67.9% 

4 

92.3% 

4 

12 

57.1% 

5 

90.4% 

5 

12 

57.1% 

8 

84.6% 

6 

12 

57.1% 

8 

84.6% 

7 

11 

60.7% 

7 

86.5% 

8 

4 

85.7% 

7 

86.5% 

9 

7 

75.0% 

7 

86.5% 
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Point  5  (No  aircraft  in  experimental  design  -  no  simulation  runs  performed.) 


ADA  Task  Force 

Run  Destroyed  Remaining _ Destroyed  Remaining 


1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 

Point  6 

Run 

Destroyed 

ADA 

Remaining 

Task  Force 

Destroyed  Remaining 

1 

5 

84.4% 

5 

90.4% 

2 

4 

87.5% 

10 

80.8% 

3 

.  4 

87.5% 

2 

96.2% 

4 

12 

62.5% 

5 

90.4% 

5 

12 

62.5% 

8 

84.6% 

6 

5 

84.4% 

4 

92.3% 

7 

5 

84.4% 

3 

94.2% 

8 

3 

90.6% 

10 

80.8% 

9 

1 

96.9% 

8 

84.6% 
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Point  7 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

5 

75.0% 

15 

71.2% 

2 

4 

80.0% 

16 

69.2% 

3 

4 

80.0% 

16 

69.2% 

4 

4 

80.0% 

16 

69.2% 

5 

3 

85.0% 

17 

67.3% 

6 

5 

75.0% 

15 

71.2% 

7 

3 

85.0% 

17 

67.3% 

8 

5 

75.0% 

15 

71.2% 

9 

5 

75.0% 

15 

71.2% 

Point  8  (No  aircraft  in  experimental  design 

ADA 

Run  Destroyed  Remaining 

-  no  simulation  runs  performed.) 

Task  Force 

Destroyed  Remaining 

1 

0 

100.0% 

0 

100.0% 

2 

0 

100.0% 

0 

100.0% 

3 

0 

100.0% 

0 

100.0% 

4 

0 

100.0% 

0 

100.0% 

5 

0 

100.0% 

0 

100.0% 

6 

0 

100.0% 

0 

100.0% 

7 

0 

100.0% 

0 

100.0% 

8 

0 

100.0% 

0 

100.0% 

9 

0 

100.0% 

0 

100.0% 
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Point  9 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

0 

0.0% 

39 

25.0% 

2 

0 

0.0% 

38 

26.9% 

3 

0 

0.0% 

39 

25.0% 

4 

0 

0.0% 

36 

30.8% 

5 

0 

0.0% 

39 

25.0% 

6 

0 

0.0% 

37 

28.8% 

7 

0 

0.0% 

39 

25.0% 

8 

0 

0.0% 

39 

25.0% 

9 

0 

0.0% 

39 

25.0% 

Point  10 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

19 

5.3% 

37 

28.8% 

2 

19 

5.3% 

39 

25.0% 

3 

16 

25.0% 

40 

23.1% 

4 

19 

5.3% 

39 

25.0% 

5 

19 

5.3% 

39 

25.0% 

6 

19 

5.3% 

37 

28.8% 

7 

19 

5.3% 

36 

30.8% 

8 

20 

0.0% 

39 

25.0% 

9 

16 

25.0% 

39 

25.0% 
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Point  1 1 


ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

8 

0.0% 

39 

25.0% 

2 

8 

0.0% 

39 

25.0% 

3 

8 

0.0% 

39 

25.0% 

4 

8 

0.0% 

38 

26.9% 

5 

8 

0.0% 

40 

23.1% 

6 

8 

0.0% 

39 

25.0% 

7 

8 

0.0% 

36 

30.8% 

8 

8 

0.0% 

40 

23.1% 

9 

8 

0.0% 

39 

25.0% 

Point  12 

ADA  Task  Force 


Run 

Destroyed 

Remaining 

Destroyed 

Remaining 

1 

8 

71.4% 

11 

78.8% 

2 

16 

42.9% 

22 

57.7% 

3 

-  15 

46.4% 

17 

67.3% 

4 

13 

53.6% 

11 

78.8% 

5 

9 

67.9% 

16 

69.2% 

6 

12 

57.1% 

11 

78.8% 

7 

13 

53.6% 

17 

67.3% 

8 

12 

57.1% 

11 

78.8% 

9 

8 

71.4% 

14 

73.1% 
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Point  13 


ADA  Task  Force 

Run  Destroyed  Remaining _ Destroyed  Remaining 


1 

10 

16.7% 

38 

26.9% 

2 

9 

25.0% 

36 

30.8% 

3 

11 

8.3% 

40 

23.1% 

4 

9 

25.0% 

38 

26.9% 

5 

10 

16.7% 

31 

40.4% 

6 

12 

0.0% 

36 

30.8% 

7 

12 

0.0% 

35 

32.7% 

8 

10 

16.7% 

32 

38.5% 

9 

9 

25.0% 

34 

34.6% 

Point  14 

Run 

Destroyed 

ADA 

Remaining 

Task  Force 

Destroyed  Remaining 

1 

9 

71.9% 

13 

75.0% 

x  2 

8 

75.0% 

11 

78.8% 

3 

6 

81.3% 

6 

88.5% 

4 

7 

78.1% 

8 

84.6% 

5 

4 

87.5% 

5 

90.4% 

6 

6 

81.3% 

8 

84.6% 

7 

6 

81.3% 

11 

78.8% 

8 

6 

81.3% 

15 

71.2% 

9 

5 

84.4% 

6 

88.5% 
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Point  15 


Run 

Destroyed 

ADA 

Remaining 

Task  Force 

Destroyed  Remaining 

1 

5 

75.0% 

9 

82.7% 

2 

3 

85.0% 

7 

86.5% 

3 

8 

60.0% 

9 

82.7% 

4 

8 

60.0% 

7 

86.5% 

5 

4 

80.0% 

2 

96.2% 

6 

3 

85.0% 

4 

92.3% 

7 

4 

80.0% 

8 

84.6% 

8 

7 

65.0% 

10 

80.8% 

9 

6 

70.0% 

8 

84.6% 

Point  16 

Run 

ADA 

Destroyed  Remaining 

Task  Force 

Destroyed  Remaining 

1 

14 

65.0% 

26 

50.0% 

2 

19 

52.5% 

20 

61.5% 

3 

.  20 

50.0% 

27 

48.1% 

4 

25 

37.5% 

27 

48.1% 

5 

14 

65.0% 

19 

63.5% 

6 

12 

70.0% 

14 

73.1% 

7 

25 

37.5% 

25 

51.9% 

8 

27 

32.5% 

32 

38.5% 

9 

20 

50.0% 

23 

55.8% 
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Center  Point 


ADA  Task  Force 


Run 

Destroyed 

Remainine 

Destroyed 

Remainine 

1 

6 

70.0% 

10 

80.8% 

2 

8 

60.0% 

6 

88.5% 

3 

9 

55.0% 

15 

71.2% 

4 

4 

80.0% 

3 

94.2% 

5 

5 

75.0% 

10 

80.8% 

6 

4 

80.0% 

4 

92.3% 

7 

8 

60.0% 

9 

82.7% 

8 

4 

80.0% 

5 

90.4% 

9 

5 

75.0% 

6 

88.5% 
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Appendix  H 


Estimated  Effects  for  the  Final  Experimental  Design 
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Estimated  Effects  and  Coefficients  for  ADA  Remaining 


Term 

Effect 

Coef 

Std  Coef 

t-value 

P 

Constant 

0.5782 

0.00665 

87.01 

0.000 

S 

0.2228 

0.1114 

0.00686 

16.27 

0.000 

L 

0.1745 

0.0872 

0.00686 

12.74 

0.000 

A 

0.3112 

0.1556 

0.00686 

22.72 

0.000 

Hp 

-0.4242 

-0.2121 

0.00686 

-30.97 

0.000 

Hd 

-0.3883 

-0.1941 

0.00686 

-28.34 

0.000 

S*L 

-0.1691 

-0.0845 

0.00686 

-12.34 

0.000 

S*A 

-0.1064 

-0.0532 

0.00686 

-7.77 

0.000 

S*Hp 

0.0526 

0.0263 

0.00686 

3.84 

0.000 

S*Hd 

0.0608 

0.0304 

0.00686 

4.44 

0.000 

L*A 

-0.1094 

-0.0547 

0.00686 

-7.99 

0.000 

L*Hp 

0.0213 

0.0106 

0.00686 

1.55 

0.123 

L*Hd 

0.0468 

0.0234 

0.00686 

3.42 

0.001 

A*Hp 

0.07 

0.035 

0.00686 

5.11 

0.000 

A*Hd 

0.0722 

0.0361 

0.00686 

5.27 

0.000 

Hp*Hd 

0.0471 

0.0235 

0.00686 

3.44 

0.001 

Analysis  of  Variance  for  ADA  Remaining 


Source 

DF 

Seq  SS 

Asj  SS 

AdjMS 

F 

P 

Main  Effects 

5 

18.2675 

18.2675 

3.65531 

535.62 

0.0000 

2-Way  Interact. 

9 

2.6224 

2.6224 

0.29138 

42.7 

0.0000 

Residual  Error 

138 

0.9418 

0.9418 

0.00682 

Curvature 

1 

0.1522 

0.1522 

0.15517 

27.03 

0.0000 

Lack  of  Fit 

1 

0.0163 

0.0163 

0.01628 

2.87 

0.0920 

Pure  Error 

136 

0.7703 

0.7703 

0.00566 

Total 

152 

21.8408 

21.8408 
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Estimated  Effects  and  Coefficients  for  the  Task  Force  Remaining 


Term 

Effect 

Coef 

Std  Coef 

t-value 

P 

Constant 

0.6978 

0.00537 

130.08 

0.000 

S 

0.1514 

0.0759 

0.00553 

13.72 

0.000 

L 

0.1132 

0.0566 

0.00553 

10.24 

0.000 

A 

0.1554 

0.0777 

0.00553 

14.06 

0.000 

Hp 

-0.3638 

-0.1819 

0.00553 

-32.89 

0.000 

Hd 

-0.2917 

-0.1458 

0.00553 

-26.37 

0.000 

S*L 

-0.0694 

-0.0347 

0.00553 

-6.28 

0.000 

S*A 

-0.0582 

-0.0291 

0.00553 

-5.26 

0.000 

S*Hp 

0.0123 

0.0061 

0.00553 

1.11 

0.269 

S*Hd 

0.047 

0.0235 

0.00553 

4.25 

0.000 

L*A 

-0.0924 

-0.0462 

0.00553 

-8.36 

0.000 

L*Hp 

0.0668 

0.0334 

0.00553 

6.04 

0.000 

L*Hd 

-0.0118 

-0.0059 

0.00553 

-1.06 

0.290 

A*Hp 

0.1058 

0.0529 

0.00553 

9.56 

0.000 

A*Hd 

-0.0198 

-0.0099 

0.00553 

-1.79 

0.076 

Hp*Hd 

-0.0315 

-0.0158 

0.00553 

-2.85 

0.005 

Analysis  of  Variance 

for  the  Task  Force 

Remaining 

Source 

DF 

Seq  SS 

Adj  SS 

Adj  MS 

F 

P 

Main  Effects 

5 

9.9868 

9.9868 

1.99737 

453.63 

0.000 

2-Way  Interact. 

10 

1.3062 

1.3062 

0.13062 

29.66 

0.000 

Residual  Error 

137 

0.6032 

0.6032 

0.004 

Curvature 

1 

0.2353 

0.2353 

0.23529 

86.97 

0.000 

Pure  Error 

136 

0.3679 

0.3679 

0.0071 

Total 

152 

11.8962 

11.8962 
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Appendix  I 

Regression  Analysis  for  the  Final  Experimental  Design 
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Regression  Analysis  for  ADA  Remaining  -  full  model 


Predictor 

Coef  Stdev 

t-ratio 

P 

Constant 

0.578169  0.006645 

87.01 

0.000 

S 

0.111411  0.006849 

16.27 

0.000 

L 

0.087233  0.006849 

12.74 

0.000 

A 

0.155603  0.006849 

22.72 

0.000 

Hp 

-0.212122  0.006849 

-30.97 

0.000 

Hd 

-0.194140  0.006849 

-28.34 

0.000 

S*L 

-0.084526  0.006849 

-12.34 

0.000 

S*A 

-0.053193  0.006849 

-7.77 

0.000 

S*Hp 

0.026304  0.006849 

3.84 

0.000 

S*Hd 

0.030397  0.006849 

4.44 

0.000 

L*A 

-0.054710  0.006849 

-7.99 

0.000 

L*Hp 

0.010634  0.006849 

1.55 

0.123 

L*Hd 

0.023406  0.006849 

3.42 

0.001 

A*Hp 

0.034980  0.006849 

5.11 

0.000 

A*Hd 

0.036097  0.006849 

5.27 

0.000 

Hp*Hd 

0.023531  0.006849 

3.44 

0.001 

s 

0.08219 

R-sq 

95.8% 

R-sq(adj) 

95.3% 

Analysis  of  Variance 

SOURCE 

DF 

SS 

MS  F 

Regression 

15 

20.9153 

1.3944  206.41 

Error 

137 

0.9255 

0.0068 

Total 

152 

21.8408 

(L*Hp  is  a  candidate  for  elimination) 


_ P 

0.000 
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Regression  Analysis  for  ADA  Remaining  -  reduced  model 


Predictor 

Coef  Stdev 

t-ratio 

P 

Constant 

0.578169  0.006679 

86.57 

0.000 

S 

0.111411  0.006884 

16.18 

0.000 

L 

0.087233  0.006884 

12.67 

0.000 

A 

0.155603  0.006884 

22.6 

0.000 

Hp 

-0.212122  0.006884 

-30.81 

0.000 

Hd 

-0.194140  0.006884 

-28.2, 

0.000 

S*L 

-0.084526  0.006884 

-12.28 

0.000 

S*A 

-0.053193  0.006884 

-7.73 

0.000 

S*Hp 

0.026304  0.006884 

-3.82 

0.000 

S*Hd 

0.030397  0.006884 

4.42 

0.000 

L*A 

-0.054710  0.006884 

-7.95 

0.000 

L*Hd 

0.023406  0.006884 

3.4 

0.001 

A*Hp 

0.034980  0.006884 

5.08 

0.000 

A*Hd 

0.036097  0.006884 

5.24 

0.000 

Hp*Hd 

0.023531  0.006884 

3.42 

0.001 

s 

0.08261 

R-sq 

95.7% 

R-sq(adj) 

95.3% 

Analysis  of  Variance 


SOURCE 

DF 

SS 

MS 

F 

P 

Regression 

14 

20.899 

1.4928 

218.74 

0.000 

Error 

138 

0.9418 

0.0068 

Total 

152 

21.8408 
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The  ( X'  X )  1  matrix  for  the  reduced  model  of  ADA  Remaining 


0.006536  0.000000  0.000000  0.000000  0.000000 

0.000000  0.006944  0.000000  0.000000  0.000000 

0.000000  0.000000  0.006944  0.000000  0.000000 

0.000000  0.000000  0.000000  0.006944  0.000000 

0.000000  0.000000  0.000000  0.000000  0.006944 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 


0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.006944  0.000000  0.000000  0.000000  0.000000 

0.000000  0.006944  0.000000  0.000000  0.000000 

0.000000  0.000000  0.006944  0.000000  0.000000 

0.000000  0.000000  0.000000  0.006944  0.000000 

0.000000  0.000000  0.000000  0.000000  0.006944 


0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.006944  0.000000  0.000000  0.000000  0.000000 
0.000000  0.006944  0.000000  0.000000  0.000000 
0.000000  0.000000  0.006944  0.000000  0.000000 
0.000000  0.000000  0.000000  0.006944  0.000000 
0.000000  0.000000  0.000000  0.000000  0.006944 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
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Regression  Analysis  for  ADA  Remaining  -  main  factors  only 


Predictor 

Coef 

Stdev 

t-ratio 

P 

Constant 

0.57817 

0.01259 

45.93 

0.000 

S 

0.11141 

0.01298 

8.59 

0.000 

L 

0.08723 

0.01298 

6.72 

0.000 

A 

0.15560 

0.01298 

11.99 

0.000 

Hp 

-0.21212 

0.01298 

-16.35 

0.000 

Hd 

-0.19414 

0.01298 

-14.96 

0.000 

s 

0.1557 

R-sq 

83.7% 

R-sq(adj) 

'  83.1% 

Analysis  of  Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

5 

18.2765 

3.6553 

150.76 

Error 

147 

3.5642 

0.0242 

Total 

152 

21.8408 

_ P 

0.000 


110 


Regression  Analysis  for  the  Task  Force  Remaining  -  full  model 


Predictor 

Coef  Stdev 

t-ratio 

P 

Constant 

0.697838  0.005365 

130.08 

0.000 

S 

0.075855  0.005530 

13.72 

0.000 

L 

0.056624  0.005530 

10.24 

0.000 

A 

0.077724  0.005530 

14.06 

0.000 

Hp 

-0.181891  0.005530 

-32.89 

0.000 

Hd 

-0.145833  0.005530 

-26.37 

0.000 

S*L 

-0.034722  0.005530 

-6.28 

0.000 

S*A 

-0.029113  0.005530 

-5.26 

0.000 

S*Hp 

0.006143  0.005530 

1.11 

0.269 

S*Hd 

0.023504  0.005530 

4.25 

0.000 

L*A 

-0.046207  0.005530 

-8.36 

0.000 

L*Hp 

0.033387  0.005530 

6.04 

0.000 

L*Hd 

-0.005876  0.005530 

-1.06 

0.290 

A*Hp 

0.052885  0.005530 

9.56 

0.000 

A*Hd 

-0.009882  0.005530 

-1.79 

0.076 

Hp*Hd 

-0.015759  0.005530 

-2.85 

0.005 

s 

0.06636 

R-sq 

94.9% 

R-sq(adj) 

94.4% 

Analysis  of  Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

15 

11.29298 

0.75287 

170.98 

Error 

137 

0.60323 

0.00440 

Total 

152 

11.89621 

(Deletion  of  any  model  terms  results  in  an  increase  in  MSE.) 


_ P 

0.000 
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The  (X}  X)  1  matrix  for  the  full  model  of  the  Task  Force  Remaining 


0.006536  0.000000  0.000000  0.000000  0.000000 

0.000000  0.006944  0.000000  0.000000  0.000000 

0.000000  0.000000  0.006944  0.000000  0.000000 

0.000000  0.000000  0.000000  0.006944  0.000000 

0.000000  0.000000  0.000000  0.000000  0.006944 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 


0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.000000  0.000000  0.000000  0.000000  0.000000 

0.006944  0.000000  0.000000  0.000000  0.000000 

0.000000  0.006944  0.000000  0.000000  0.000000 

0.000000  0.000000  0.006944  0.000000  0.000000 

0.000000  0.000000  0.000000  0.006944  0.000000 

0.000000  0.000000  0.000000  0.000000  0.006944 

0.000000  0.000000  0.000000  0.000000  0.000000 


0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.006944  0.000000  0.000000  0.000000  0.000000 
0.000000  0.006944  0.000000  0.000000  0.000000 
0.000000  0.000000  0.006944  0.000000  0.000000 
0.000000  0.000000  0.000000  0.006944  0.000000 
0.000000  0.000000  0.000000  0.000000  0.006944 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 
0.000000  0.000000  0.000000  0.000000  0.000000 


0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.000000 

0.006944 
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Regression  Analysis  for  the  Task  Force  Remaining  -  main  factors  only 


Predictor 

Coef  Stdev 

t-ratio  p 

Constant 

0.697838  0.009214 

75.74 

0.000 

S 

0.075855  0.009497 

7.99 

0.000 

L 

0.056624  0.009497 

5.96 

0.000 

A 

0.077724  0.009497 

8.18 

0.000 

Hp 

-0.181891  0.009497 

-19.15 

0.000 

Hd 

-0.145833  0.009497 

-15.36 

0.000 

s 

0.114 

R-sq 

83.9% 

R-sq(adj) 

83.4% 

Analysis  of  Variance 

SOURCE 

DF 

SS 

MS 

F 

Regression 

5 

9.9868 

1.9974 

153.77 

Error 

147 

1.9094 

0.0130 

Total 

152 

11.8962 

_ P 

0.000 
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Appendix  J 

Residuals  Plots  for  the  Final  Experimental  Design 
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Normal  Probability  Plot  of  Residuals  for  the  ADA  Remaining 


Normal  Probability  Plot 


Average: -0.0000000 
Std  Dev: 0.0780305 
Not  data:  153 


Residuals  for 
ADA  Remaining 


Kolmogorov-Smimov  Normality  Test 
D+:  0.199  0-:  0.168  D  :  0.199 
Approximate  p  value  <0.01 


Normal  Probability  Plot  of  Residuals  for  the  Task  Force  Remaining 


Normal  Probability  Plot 


ara  uev:  u.uo^aaoo  ■  . . a  u+:u.iv^  u-;u.no  u.u.ia. 

N  of  data:  153  Approximate  p  value  <0.01 


Both  plots  show  that  the  residuals  generally  fall  along  the  line  of  normal  probability. 
Each  plot  also  includes  the  results  of  a  Kolmogorov-Smimov  (K-S)  test  for  goodness  of 
fit  on  the  normal  distribution.  Neither  the  plots  or  the  K-S  tests  provide  sufficient 
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evidence  to  doubt  the  assumption  of  normality.  See  below  for  a  discussion  of  the  K-S 
test. 

The  K-S  test  is  a  goodness  of  fit  test  to  determine  if  sample  data  fits  a  hypothesized 
distribution  (Law  &  Kelton).  The  test  takes  several  forms  based  on  the  knowledge  of  the 
hypothesized  distribution.  For  the  case  here,  we  reject  the  hypothesis  that  the  sample 
data  approximates  the  normal  distribution  when: 


Vo.oi+“f 

v n , 


Dn  >  C\-a 


where: 

c'j^  =  0.895  at  a  =  0.05. 

Dn  =  max  {Dn+,Z); 

/  =  the  rank  order  position  of  the  observation  (lowest  to  greatest) 

F(Xi )  =  the  value  of  the  cumulative  probability  distribution  of  the  hypothesized 
(normal)  distribution.  X,  here  is  the  value  of  the  residual.  The  residual 
sample  mean  and  variance  are  used  as  estimates  for  the  distribution  mean 
and  variance. 
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Plot  of  Residuals  versus  the  Fitted  Values  of  ADA  Remaining 


Plot  of  Residuals  versus  the  Fitted  Values  of  Task  Force  Remaining 


Both  of  the  above  plots,  indicate  that  the  assumption  of  constant  error  variance 
appears  to  hold  true  for  the  center  of  the  design  space.  The  small  variances  shown  at  the 
“tails”  of  the  responses  are  the  result  of  the  experimental  design  and  the  definitions  of  the 
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responses.  It  is  impossible  for  actual  response  values  to  exceed  100%  or  be  lower  than 
0%.  However,  the  regression  equations  could  conceivably  predict  values  that  would 
violate  these  logical  extremes.  In  these  situations,  the  larger  predicted  values  would 
correspond  to  observed  values  that  lie  on  or  near  0%  or  100%.  Hence,  this  negates  the 
possibility  of  large  variances  at  the  extremes. 


Plots  of  residuals  versus  factors  provide  information  regarding  model  specification. 
A  plot  showing  a  “curve”  points  toward  the  nonlinearity  of  the  factor  plotted  versus  the 
residuals  (Fox).  The  following  pages  contain  plots  of  the  residuals  versus  each  of  the 
five  main  factors  of  the  two  response  models. 
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Residuals  for 
ADA  Remaining 


Plots  of  Residuals  versus  Each  Main  Factor  for  ADA  Remaining 


n  i  - - r 

-i  o  i 

S 


-i  o  i 

L 
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Residuals  for 
ADA  Remaining 


120 


Each  of  the  above  residual  plots,  for  the  Air  Defense  Remaining,  suggest 
nonlinearity  in  the  main  factors.  It  is  difficult  to  determine  which  factor  or  factors  are 
truly  nonlinear  as  there  is  only  one  center  point  in  the  design. 
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Plots  of  Residuals  versus  Each  Main  Factor  for  the  Task  Force  Remaining 
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As  with  the  plots  for  the  Air  Defense  Remaining,  each  of  the  residual  plots,  for  the 
Task  Force  remaining,  suggest  nonlinearity  in  the  main  factors.  It  is  difficult  to 
determine  which  factor  or  factors  are  truly  nonlinear  as  there  is  only  one  center  point  in 
the  design. 
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Appendix  K 

Residuals  for  the  Final  Experimental  Design 
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For  identifying  possible  outliers  and  points  of  high  influence  (leverage  points),  the 
following  relationships  are  used: 

1.  Residual  -  the  difference  between  the  observed  value  and  predicted  response. 

2.  St.  Residual  (r.  )  -  the  internally  studentized  residual,  sometimes  referred  to  as  the 

standardized  residual. 


Mi -A,) 

where: 

a 2  is  estimated  by  MSE 
e.  is  the  ordinary  residuals 
h..  is  the  /th  diagonal  of  H  (the  hat  matrix) 
Observations  with  |  r;  |  >  2  are  listed  in  the  tables  below. 

3.  R-Student  (/,.)-  the  externally  studentized  residual 


/,  = 


e. 


JsfQ-K) 


where: 

ei  is  the  ordinary  residuals 

hH  is  the  /'th  diagonal  of  H  (the  hat  matrix) 


s: 


2  _  (n-p)MSE-ef  /(\-hu 


n-p-l 

Observations  with  |  /;  |  >  2  are  possible  outliers  (Fox). 


4.  Cook’s  D  (D, )  -  Cook’s  distance 


D,  = 


K 


KP) 


where: 

r2  is  the  square  of  the  ordinary  residual 
p  is  the  number  of  parameter  in  the  model 
htt  is  the  /th  diagonal  of  H  (the  hat  matrix) 

Observations  with  Z).  >  4 /(« -p  - 1)  are  possible  leverage  points  (Fox). 
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Residuals  of  the  Final  Model  with  Interactions  for  ADA  Remaining 


Observation  Data  Point 

Residual 

St.  Resid. 

R-Student 

Cook's  D 

35 

4 

0.19172 

2.45147 

2.49756 

0.04638 

49 

6 

-0.19525 

-2.49651 

-2.54560 

0.04810 

50 

6 

-0.19525 

-2.49651 

-2.54560 

0.04810 

114 

13 

-0.16674 

-2.13207 

-2.16021 

0.03508 

115 

13 

-0.16674 

-2.13207 

-2.16021 

0.03508 

141 

16 

0.19156 

2.44940 

2.49536 

0.04630 

143 

16 

-0.18344 

-2.34556 

-2.38507 

0.04246 

148 

center 

0.22183 

2.69409 

2.75782 

0.00318 

149 

center 

0.17183 

2.08685 

2.11288 

0.00191 

150 

center 

0.22183 

2.69409 

2.75782 

0.00318 

152 

center 

0.22183 

2.69409 

2.75782 

0.00318 

153 

center 

0.17183 

2.08685 

2.11288 

0.00191 

The  D,  cutoff  for  the  ADA  Remaining  response  model  is  0.02899.  Observations  35,  49, 
50,  1 14,  115,  141,  and  143  all  exceed  the  cutoff. 


Residuals  of  the  Final  Model  with  Interactions  for  the  Task  Force  Remaining 


Observation  Data  Point 

Residual 

St.  Resid. 

R-Student 

Cook's  D 

4 

1 

-0.20639 

-3.29819 

-3.42491 

0.08463 

101 

12 

-0.15510 

-2.47866 

-2.52691 

0.04780 

141 

16 

0.17609 

2.81410 

2.88854 

0.06161 

143 

16 

-0.17006 

-2.71769 

-2.78384 

0.05746 

146 

center 

0.18678 

2.82402 

2.89935 

0.00328 

148 

center 

0.24447 

3.69632 

3.88143 

0.00562 

150 

center 

0.22524 

3.40555 

3.54654 

0.00477 

152 

center 

0.20601 

3.11478 

3.21949 

0.00399 

153 

center 

0.18678 

2.82402 

2.89935 

0.00328 

The  D.  cutoff  for  the  Task  Force  Remaining  response  model  is  0.02920.  Observations 
4,  101,  141,  and  143  exceed  the  cutoff. 
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Observations  101,  141,  and  143  are  the  only  three  observations  that  are  leverage 
points  for  both  response  models.  Although  these  leverage  points  are  also  possible 
outliers,  the  data  reflects  the  stochastic  nature  of  the  simulation.  After  verifying  all 
simulation  parameters,  it  is  determined  that  the  observations  must  remain  included  in  the 
experiment  to  account  for  variability  in  the  model. 
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Appendix  L 


Air  Defense  Force  Allocation  Planning  Tool 
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The  Air  Defense  Force  Allocation  Planning  Tool  consists  of  a  series  of  linked 
spreadsheets.  Built  in  Microsoft  Excel,  the  tool  has  a  very  simple  design  and  easy  to  use 
interface.  The  Data  Entry  screen,  shown  below,  requires  the  user  to  enter  values  for  the 
expected  enemy  threat.  For  this  prototype  tool,  the  Stinger  weapon  system  is  selected  to 
remain  constant  throughout  the  analysis.  The  uncomplicated  design  of  the  tool  facilitates 
expanding  this  selection  to  include  the  other  two  ADA  weapon  systems  considered, 
Linebackers  and  Avengers. 


m  Microsoft  Excel  -  TOOL.XLS 


The  Task  Force  Remaining  screen  (TF  Rem)  and  the  ADA  Remaining  screen  (ADA 
Rem)  show  the  predicted  percentages  of  the  amounts  remaining  of  the  task  force  and 
ADA  assets.  The  predictions  are  based  on  the  regression  equations  developed  for  each 
response  model,  Equations  2  and  3  in  the  Data  Analysis  chapter.  The  TF  Rem  screen 
appears  below.  The  ADA  Rem  screen  is  nearly  identical  in  appearance  to  the  TF  Rem 
screen. 
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The  Task  Force  Half  Width  of  the  Prediction  Interval  screen  (TFint)  and  the  ADA 
Half  Width  of  the  Prediction  Interval  screen  (ADAint)  show  the  factors  that  must  be 
added  and  subtracted  to  the  predicted  responses  to  form  90%  prediction  intervals.  The 

i 

prediction  interval  half  widths  are  based  on  the  prediction  interval  equation  found  in  the 
Application  and  Limitations  chapter,  Equation  6.  For  easy  reference,  the  set  up  the 
prediction  interval  half  width  screens  is  identical  to  the  set  up  of  the  screens  for  the 
predicted  responses.  The  TFint  screen  is  shown  below;  the  ADAint  screen  appears 
nearly  identical. 
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Each  prediction  and  interval  factor  sheet  contains  links  to  the  Data  Entry  sheet.  The 
links  make  available  the  Stinger,  HIP  and  HIND  values  needed  for  each  calculation.  At 
upper  left  hand  comer  of  each  data  table,  an  equation  cell  contains  the  equation  for  the 
regression  model  or  appropriate  prediction  interval  factor.  This  comer  cell  is  usually 
hidden  from  view  when  the  tool  is  in  use.  The  ADA  Rem  screen  below  shows  all  hidden 
cells  and  the  calculations  contained  in  the  equation  cell.  The  other  spreadsheet  screens 
are  similar  in  design. 
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